Neal Waterstreet

Hadoop training Day 3

by Leave a Comment

Training this week is starting to go by quickly. Today we’ll be covering these topics:

  • Hive programming
  • ngrams
  • HCatalog

Even in a limited testing environment it is easy to see the benefits of Hive. We also looked at a comparison of Hive versus SQL. For a couple of the labs we used ngrams to search email data. Towards the end of the session we discussed HCatalog, which is the central schema repository.

As I mentioned earlier, this week is going by quickly and I know we are just scratching the surface of some of these topics. This class is the same as almost all IT training, there will be a lot of work to do on my own to get the most out of this.

Hadoop training Day 2

by Leave a Comment

Day two of my training was all about Pig, which is a scripting language for exploring data sets.

Here is a list of a few topics we covered today:

  • Pig Latin
  • A few operators
  • Grunt
  • Bag
  • Joins

We also worked on a few examples covering clickstream and stock market data.

I’m starting to realize how much there really is to learn here. We’re cramming a lot of information into each session, but I think this is the best way to get some experience with it.

Hadoop training Day 1

by Leave a Comment

I decided it is finally time to jump in and start looking at Hadoop. I’d originally scheduled for Windows version of the class, but it was cancelled so I switched to the Apache version. At this point, I really wish I’d kept up with Linux commands…

Some of the topics we have discussed so far:

  • Overall Hadoop architecture
  • HDFS
  • YARN
  • MapReduce
  • Briefly discussed Pig, but will get into that more tomorrow

I’m taking the Hortonworks official training since they are a Microsoft partner. One thing that is a little odd is that I am the only one in my classroom. Everyone else (about 15) are in different cities/offices. I really set up with combination of cameras and WebEx. It seems to work well, at least so far. The learning facilitator is top notch as well. Definitely not his first time teaching the course.

Lots to learn with Hadoop and I am hoping this course will help me get up and running.