Source and Wiki

A new url for everyone who wants to keep up-to-date:

Hackystat Wiki and SVN

The SVN should be readable so people can checkout a read-only copy and hopefully compile for there own use.

Busy week

It’s been a very busy week. Aside from my own personal life I was able to get some data sent all the way from VSTFS to the development Hackystat database we have setup at UofT. It uses the pre-existing sensorshell to send data which gives the benefit of offline storage and product longetivity with a con of being slower than just making my own. Production has been good and the sensor now sends alot more data about version control than is shown in the screenshot below. I hope to utilize as much of the data as is provided by VSTFS event system to send to Hackystat and maybe even some that isn’t 😉 . Right now it easily sends the equivalent amount of information as the SVN sensor. I shall provide screenshots next week.

In case you were still wondering what VSTFS is exactly, its basically some super-powered team management software and includes all of the tools needed to facilitate team coordination and planning. Its like the corporate edition of DrProject. Right now the tool senses version control info but has the ability to sense more I just need to decide what I want to send and what is overhead info.

One of the most frustrating things about developing a sensor for such a large and requirements dependent system is the setup. Thats why I plan to make the setup and configuration of this sensor as easy and foolproof as possible. On that note, I am done about 70% of the initial settings infrastructure interface that will allow users of this very large sensor to quickly and easily configure critical sensor properties and store them in the (required) running MSSQL Server that stores the other various VSTFS information. Since all settings need to be stored in an sql database there would need to be some heavy security authentication going on if the user wishes to access the sensor properties outside the local system. Therefore I decided that at the moment it can only be run and changed from the computer taht is running VSTFS. I hope to change that in the future… sometime… maybe… we will see.

I am finally at that point in the project where you (and possibly your team) have built up a codebase large enough that you can actually see components working together and really begin to see how the system is functioning and what needs improvement. Its been a bit slow and I’ve had to brush up on my SQL (especially for database creation) as well as SOAP and XML. With a good foundation now its hard to choose from the tons of cool ideas I have on how to extract more data and how to run things more efficiently. Overall its proving to be a really good learning experience.

Advertisement

First data sent!

Today the service sent the first data ever. The data was very simple but its still cool that it works.

Don’t get to happy yet though. The code is all over the place and a ton of features, methods, properties, functions, etc are missing or need heavy refactoring. It definitely helped that there was already a wrapper class for the visual studio plugin that I could use. It may be possible to build further upon the existing code to make it easier for other people developing sensors in c#.

Woot for progress!

So today I finished the subscription part of the application. I wrote a small app that would determine what events were available (unfortunately those are manually imported into an xml ATM), displays which events are currently be registered by querying the database, and also allows for manipulation of registred events (subscribing/unsubscribing) using checkboxes. The interface is mediocre but the whole system works very nicely and can be used to easily get the VSTFServer sending the specified events to the services defined in the TFS Notification Web Services Template. There is a ton of testing and a few hard-coded items that must be changed later; but for now step 3 is basically complete.

More of a personal reminder to myself than anything to report. It may be worth while looking at the possibility of using polling to extract other information that is not reported using a webservice by VSTFS. It would be slower but that database has a TON of info in it and its only just waiting for me to extract it. (BTW: SQL is teh awesome!)

The next step (goal for next week) is to build a simple hackystat connecter that will allow for data collected through the webservice to be sent to a hackystat sensorbase. This may be a bit trickier because the various events all collect different types of data so writing a unified general adapter may be a bit tricky if not impossible.

So many problems

Yesterday there was a massive bug in the eclipse plugin for Hackystat. Somehow it managed to record over 30GBs of data!!! Unfortunately this crashed the server which only had a 50GB drive most of which was filled with a large virtual machine and the client using the plugin who also had a small drive which ran out of space bringing their system to a halt. The problem has been fixed and the Hackystat team has been made aware but its a little discouraging. Hopefully during the course of this GSoC project I won’t encounter too many more bugs :p .

In other news I have begun work on the actual sensor and am currently in the midsts of writing a web-interface to allow for some basic sensor configuration and VSTFS connection items before I actually write the code to connect to the hackystat server. I am doing a ton of re-learning since I last used C# and ASP about 8 months ago. Overall though its pretty cool and I can’t wait till the first pieces of data start flying towards the Development Hackystat server.

P.S. When developing open-source software the key to reeling in the vast majority of clients is to develop software that is easy to install/update/repair/re-install/remove. Very few “average Joe’s” want to build your system from source, configure the environment variables, change the settings files, install dependancies… etc, etc, etc. As you can tell I have done way to much installing/repairing recently.

The current plan

So if anyone is wondering what exactly is going through my head at the moment, don’t worry, I have a fool proof plan…

<Step #> <Codename>: Description

1. Alpha: Setup the environment. (Complete)
2. Two’s company: Basic research into the architecture and design of a VSTFS app. (Complete)
3. Three’s a Crowd: Design a simple app that can listen for a small subset of the available events triggered by TFS.
4. The Fantastic Fourth: Design a simple app that can send data to a Hackystat server. May include looking at current visual studio sensor for Hackystat.
5. 5th element: Put simple apps together to get a beta app for the mid-point of GSoC.
6. Sixth Sense: Advanced research into the more obscure events/information I can get from VSTFS to send to Hackystat.
7. Lucky seven: Build on the simple apps to increase functionality.
8. Because 7,8,9: Begin work on some unit-tests and the documentation as a Google code-project.
9. Cats: Create an easy to use installer.
10. Decimal: Bug-testing.
11. Hour: Final clean-up and code submission.
12. Hex: Sit back and relax).

So thats the general idea. Obviously some parts may take longer than others but hopefully nothing will be too simple or too challenging.

With progress there is change!

So yesterday and today I delved into samples from the Visual Studio Team Foundation Server 2005 SDK which is a subset of the Visual Studio 2005 SDK. My goal was to investigate some way of listening to TFS (Team foundation server) events as outlined in the documentation I had been reading during the massive installation I had been working on. I learned two things (one of which I had already suspected since the beginning):

1. The “Team Foundation Server” does not directly allow for plugins/add-ins to be added (from what I have read and seen). I suspected this already and therefore prepared myself for some client<->server programming.
2. The existing architecture for listening to Events is web-based, which is still client-server and involves me programming in c# but may require some additional setup for whomever wishes to install the sensor in the near future.

I didn’t expect number 2 but after about an hour of searching around the web I found a nifty bit of code template to confirm my theory.

• TFS Notification Web Services Template

It took me awhile to get it running as I had never used anything but the default empty template before but after some more poking around the web and fiddling with using BisSubscribe to tell the server I was listening for events I was able to get this:

So now that a have a starting point for next week that will focus on building a small web-app that listens to some events and has some customizability. Overall its looking pretty good so far.

Language choice?

I had originally planned to design the VSTFS Sensor in Java to keep with the current motif of Hackystat. The more I look at the VSTFS SDK the more I think it will need to be C#. The more I think about it this may also be beneficial since I will be actively testing the sensor while I am coding since I will most likely be using VS2005 to code if using C#.

Setting up Hackystat

First I suggest everyone reading this go find out “What is Hackystat “.

Last week I and my partner setup Hackystat on Linux (Ubuntu Hardy Heron 8.04 specifically). It toke quite a bit longer than I had expected but we finally got it all up and running as root on the machine. The documentation for Hackystat is very thorough which makes it easier to go through but there is also alot of it which takes some time. I definitely have a much better feel for the system and how it works as a whole. Unfortunately the sensor service is not at a publicly available address so you wont be able to view it, but you can view the main server at Hackystat Hawaii.

Beyond the realm of Hackystat I also had to setup:

• A virtual machine image of Windows Server 2003 Enterprise edition.
• Updates for windows (over 2GBs)
• Windows IIS Server
• Microsoft SQL Server 2005 Trial (6 months)
• (Visual Studio Team Foundation Server) VSTFS 2005 Trial (180 Days)
• Visual Studio 2005 (Pending as of today because of issues with VSTFS)

Needless to say this was very log process but luckily I had lots of documentation to go through:

• Hackystat Documentation
  
• Visual Studio Team Foundation Server SDK

I expect I will be going through more documentation this week as well. I hope to have some screenshots of the whole system working by the end of the week along with my progress on building a simple VSTFS plugin/add-in/service/”whatever they are calling it now”.

Also some info about using Hackystat at UofT for GSoC: Using Hackystat

P.S. Philip Johnson tells us the weather in Hawaii is amazing, the weather is Canada, is well, less amazing.