Friday 8 March 2013

How does a NameNode handle the failure of the data nodes?


HDFS has master/slave architecture. An HDFS cluster consists of a single NameNode, a master server that manages the file system namespace and regulates access to files by clients.

In addition, there are a number of DataNodes, usually one per node in the cluster, which manage storage attached to the nodes that they run on.

The NameNode and DataNode are pieces of software designed to run on commodity machines.
NameNode periodically receives a Heartbeat and a Block report from each of the DataNodes in the cluster. Receipt of a Heartbeat implies that the DataNode is functioning properly.

A Blockreport contains a list of all blocks on a DataNode. When NameNode notices that it has not received a heartbeat message from a data node after a certain amount of time, the data node is marked as dead. Since blocks will be under replicated the system begins replicating the blocks that were stored on the dead DataNode.

The NameNode Orchestrates the replication of data blocks from one DataNode to another. The replication data transfer happens directly between DataNode and the data never passes through the NameNode.

28 comments:

  1. Managing a business data is not an easy thing, it is very complex process to handle the corporate information both Hadoop and cognos doing this in a easy manner with help of business software suite, thanks for sharing this useful post….
    Regards,
    cognos Training in Chennai|cognos Training Chennai|cognos Training

    ReplyDelete
  2. Maharashtra Police Patil Recruitment 2016

    Thanks for sharing it. I found this Information very interesting and informative Keep sharing.......

    ReplyDelete
  3. I simply wanted to write down a quick word to say thanks to you for those wonderful tips and hints you are showing on this site.

    Best Java Training Institute Chennai



    Amazon Web Services Training in Chennai


    ReplyDelete
  4. You’ve written a really great article here. Your writing style makes this material easy to understand.. I agree with some of the many points you have made. Thank you for this is real thought-provoking content
    python training in chennai | python training in bangalore

    python online training | python training in pune

    python training in chennai | python training in bangalore

    python training in tambaram | python training in velachery

    ReplyDelete
  5. This is most informative and also this post most user friendly and super navigation to all posts... Thank you so much for giving this information to me.. 
    python online training
    python training in OMR
    python training institute in chennai

    ReplyDelete
  6. nice article... lot of information... thank you for sharing the info for any online training like micro dynamics crm training online Microsoft Dynamics CRM Training Online and many other step to this.

    ReplyDelete
  7. I wondered upon your blog and wanted to say that I have really enjoyed reading your blog posts. Any way I’ll be subscribing to your feed and I hope you post again soon.
    German Classes in Chennai
    Java Training in Chennai
    German language training in chennai
    German Training in Chennai
    Java course in Chennai
    Java Training Institute in Chennai

    ReplyDelete
  8. Thanks for the informative article. This is one of the best resources I have found in quite some time. Nicely written and great info. I really cannot thank you enough for sharing.
    Java training in Bangalore | Java training in Marathahalli | Java training in Bangalore | Java training in Btm layout

    Java training in Bangalore | Java training in Jaya nagar | Java training in Bangalore | Java training in Electronic city

    ReplyDelete
  9. I would assume that we use more than the eyes to gauge a person's feelings. Mouth. Body language. Even voice. You could at least have given us a face in this test.
    Data Science Training in Indira nagar
    Data Science training in marathahalli
    Data Science Interview questions and answers


    ReplyDelete
  10. Thank you a lot for providing individuals with a very spectacular possibility to read critical reviews from this site.
    python course institute in bangalore | python Course institute in bangalore| python course institute in bangalore

    ReplyDelete
  11. I’ve got some recommendations for your blog you might be interested in hearing.\
    fire and safety course in chennai

    ReplyDelete
  12. Hmm, it seems like your site ate my first comment (it was extremely long) so I guess I’ll just sum it up what I had written and say, I’m thoroughly enjoying your blog. I as well as an aspiring blog writer, but I’m still new to the whole thing. Do you have any recommendations for newbie blog writers? I’d appreciate it.


    AWS Training in Bangalore | Amazon Web Services Training in bangalore , india

    AWS Training in pune | Amazon Web Services Training in Pune, india

    AWS Training in Chennai|Amazon Web Services Training in Chennai,India


    aws online training and certification | amazon web services online training ,india

    ReplyDelete
  13. You made such an interesting piece to read, giving every subject enlightenment for us to gain knowledge. Thanks for sharing the such information with us
    devops online training

    aws online training

    data science with python online training

    data science online training

    rpa online training

    ReplyDelete
  14. After seeing your article I want to say that the presentation is very good and also a well-written article with some very good information which is very useful for the readers....thanks for sharing it and do share more posts like this.
    Microsoft Azure online training
    Selenium online training
    Java online training
    uipath online training
    Python online training


    ReplyDelete
  15. Your very own commitment to getting the message throughout came to be rather powerful and have consistently enabled employees just like me to arrive at their desired goals.
    Data science Course Training in Chennai | Data Science Training in Chennai
    RPA Course Training in Chennai | RPA Training in Chennai
    AWS Course Training in Chennai | AWS Training in Chennai

    ReplyDelete
  16. Thanks for sharing such informative guide on .Net technology. This post gives me detailed information about the .net technology. I am working as trainer in leading IT training academy offering Dot Net Training in Chennai and i use your guide to educate my students.




    Dot Net Training in Chennai | Dot Net Training in anna nagar | Dot Net Training in omr | Dot Net Training in porur | Dot Net Training in tambaram | Dot Net Training in velachery

    ReplyDelete
  17. If AWS is a job that you're dreaming of, then we, Infycle are with you to make your dream into reality. Infycle Technologies offers the best AWS Course Chennai, with various levels of highly demanded software courses such as Oracle, Java, Python, Hadoop, Big Data, etc., in 100% hands-on practical training with specialized tutors in the field. Along with that, the pre-interviews will be given for the candidates, so that, they can face the interviews with complete knowledge. To know more, dial 7502633633 for more.Learn AWS Course in Chennai | Infycle Technologies

    ReplyDelete