Neal Waterstreet

Presenting at Atlanta Microsoft Business Intelligence Group

by Leave a Comment

I’m presenting R 101 (Introduction to R) at the Atlanta Microsoft Business Intelligence Group on Monday, March 30th. It is the first time I’ve presented in a year and a brand new topic (for me) – R.

Here is the abstract:

“Even though it has been around for a long time, R is still one of the leading languages for data science. It has recently started to attract a lot of attention, but what is R and what is it used for? This session will help with getting started using R by looking at the basics of the language including: reviewing the basics of the R syntax, leveraging existing packages, importing and modifying data and visualizing the results.”

If you get a chance, please try to stop by.

VoIP with OBi100

by Leave a Comment

We decided to cut the cord a while back and overall that has worked out fine. One of the things I found that I missed was having a phone at the house. Its also convienient to have a home phone number instead of using a mobile one. Using a VoIP service seemed like a good solution, but there were always some issues with it.

I read about the ObiHai ATAs and selected the OBI100. It was inexpensive, but the deciding factor was that the ObiHai devices all work with Google Voice. I think ObiHai has a VoIP service you can use, but the flexibility of having another Google Voice number was too much to pass up.

You can read about the setup here, and it was very simple. It took just a few minutes to get installed and configured.

I am really happy with it so far. It seems like a good solution for those of us that no longer use analog phones.

Problem with SCD

by Leave a Comment

Problem

A common (and very well documented) best practice for performance is to specify the columns in a SQL statement instead of using SELECT *. This applies to SSIS in the Data Flow Task, but there are other reasons to avoid this too.

At a recent client the current member row was being recreated daily for a SCD 2 dimension. The data in the source rarely changed, so it was obvious that something was wrong.

My first step was to look at the code, including the Conditional Split task. All of the logic there looked correct – all of the comparisons appeared to be functioning as intended and NULL values were being accounted for in the code.

I then checked some of the columns to determine if there was a problem with spaces by using the RTRIM() and LTRIM() functions. That reminded me that there is still not a TRIM() function. All of this still looked to be functioning correctly.

Finally I checked the Source and Destination Connection Managers and found they were configured to use tables as a source. The end result of this was that the current member rows were not filtered in the Destination and all rows were incuded.

Solution

I changed the Destination SQL statement to select only the current member rows selected in the WHERE clause. This resolved the issue and the SCD 2 no longer created duplicate rows.

Hadoop training Day 3

by Leave a Comment

Training this week is starting to go by quickly. Today we’ll be covering these topics:

  • Hive programming
  • ngrams
  • HCatalog

Even in a limited testing environment it is easy to see the benefits of Hive. We also looked at a comparison of Hive versus SQL. For a couple of the labs we used ngrams to search email data. Towards the end of the session we discussed HCatalog, which is the central schema repository.

As I mentioned earlier, this week is going by quickly and I know we are just scratching the surface of some of these topics. This class is the same as almost all IT training, there will be a lot of work to do on my own to get the most out of this.