Lessons Learned from GSoC 2008 (or in other words, the final post)

I learned a ton this summer during my GSoC project. Some of the more important things I learned were…

• People in the open-source community are always willing to help if they can.
• Make sure you figure out what the prerequisites of an API are before you use them.
• Hackystat
• If you have a specific GSoC project in mind, go with it but expect that a least one or more critical things you wanted to do in your project will get changed or entirely scraped. (Oh my poor easy auto installer)
• WRITE GOOD CODE AND DOCUMENTATION. There is tons of bad code out there and most of it is undocumented so refactor and compile those code documents.
• Making screencasts is easy but doing the voice-over is really hard.

Overall GSoC was a pretty awesome experience and I met a ton of cool people an learned alot of solid and useful things. My only reget is of course that my program was not as polished as it could have been.

Some techinicall stuff I learned was:

• Team Foundation Server!!!
• A better understanding of c# to say the least
• A bit about how SOAP and the REST API work
• Some web services stuff

Advertisement

Its high time

Its high time you checked out the Hackystat TFS Sensor Google Project Wiki. I created a little demo showing off some of the functionality of the sensor. I think its pretty good. Suggestions are welcome. Also if anyone knows how to integrate it into a wiki page that knowledge would be awesome.

…

So this week I worked on the wiki a bit more to make it more thorough. Needless to say I also fixed up a few bugs.

I also worked with Luke Luke Petrolekas for a bit of usability and error testing. He was able to point out a few errors and places where I need to have a more through install guide. We also discovered that the problem (in the post below) also does occur on 32-bit systems. Once this error has been fixed I will be able to release version 1 of the application. Otherwise I will have to release version 0.9 which must be run under specific conditions. I would hate to have to do that.

This week, that week, any old week.

If anyone is keeping track I am on something like step “11. Hour: Final clean-up and code submission.” of my 12 step plan. I skipped step “9. Cats: Create an easy to use installer.” because it would require me dishing out about $20 for a decent installation creator that automagically installs most prerequisites. A project in itself. On to more important things.

This week I was trying desperately to get the sensor working on a production server. Mainly my home computer outside of the virtual machine. Most things went ok, there are still some bugs, and boy oh boy you better set your file permissions correctly. I am going to try to get someone else to test it again (where is Luke…) but I need to sort out one very annoying bug. There is a problem when I run the sensor outside of the development environment computer. No matter what I do, every time I try and access a TFS API function I get something similar to this:

Very descriptive right. Looking around the net gives two possibilities and ONLY two possibilities.

• The compiler compiled a very large program with to many variables. … NOPE!
• The TFS API does not compile for x64 systems. Maybe…

My system at home runs on an Intel x64 processor but I am specifically running Windows XP Pro 32-bit instead of x64 to avoid such issues. Either way I am pretty sure its not my fault, and something is going strange with the compiler. I even added an “if (tfs != null)” statement, which CRASHED! How the heck do you crash on that. It’s either null or its not null. Sheesh, maybe there really is a third-bit. I’ll try testing it on a strictly x86 system next week.

Looking at the code right now I think some of it is good and well… lots bad. It all works pretty flawlessly in development but who knows about a release server. Therefore I am trying to break it as hard as I can. I also need to add in some more GUI prompts when things fail (like trying to find the TFS server or loading the sensorshell). Everything is logged in the log files of course, but who wants to check that every three seconds. And what happens if the log file cannot be written because of say… permissions?

I also migrated all the code and most of the documentation over to Hackystat TFS on Google Code. I think it will be happier there as it will allow more people to help contribute to it. It also better fits the Hackystat motif. This means that I will be deleting all my code and documentation from the Dr. Project site so others don’t get confused later. BTW: Anyone know how to limit image sizes with the Google Wiki Syntax?

I also need to make a screencast video showing of my awesome sensor working. Any tips (programs, editing) on that would be very helpful.

Near the end

Well there are only two weeks left according to the GSoC timeline. That means its time to hunker down and start testing on a production server and write some documentation. Thats what I started doing on the end of this week. Most of the week though was spent using the TFS API to try and extract additional info from the server. For example I was able to extract the number of lines added, changed, and deleted for a revision compared to its previous revision. The result can be given in a variety of different formats but all of them are plain text (not objects or datatypes) which I think is pretty silly. Therefore I choose for the API to deliver the goods in unix normal format and wrote a small parser to identify the valuable info. Its pretty bad but according to the Hackystat commit datatype, having the number of lines added and deleted is required so I put it in.

I also figured out how to use the API to trigger certain events such as builds and commits. I later found out its pretty much usless for testing though since its near impossible to test an asynchronous item with synchrnous unit tests. For example if I commit a “test” item to the repository how long do I wait for a response back, what if it never comes, is it even possible to wait for a response in a unit test environment.

Though learning how the API functions was not in vain. Most of the code and knowledge I learned was used for imporving upon previous features that I had been doing in a sort of round about way. For example, I was using the BisSubscribe.exe tool to subscribe to VSTFS events. This has bad idea written all over it but I didn’t want to mess directly with the pre-created tables in the VSTFS MSSQL server directly in fear that I may cause some horrendous error, not to mention all the security issues. Luckily while learning the API I found there was a class that was used for just this purpose. Since my code is pretty modular I simply wrote another method which used this service and then gave the user the option of selecting between subscribing using the eventing service API or the BisSubscribe. I think its a good compromise.

I am not MIA

I kindof forgot to post an update last week. Sorry all. Then I got sick this week for a few days which hampered my progress somewhat. Not to worry though. I didn’t just sit around doing nothing I actually did some reading. At a university industry lunch  I was given the book “Professional Team Foundation Server” book by Jean-Luc David who co-wrote. It really has a lot of good material for using and extending Visual Studio Team Foundation Server. After I recovered a bit from my cold/flu/bug I began to write some Unit Tests that incorporated some of the things I had learned from the book. For example I really wanted to be able to programming check-in things to source control or work items so I could truly test the system instead of simply using mock data. I haven’t worked out all the bugs for the new Unit Tests this week so they are not in source control yet but hopefully but Tuesday next week.

Also sometime within the past two weeks I was able to fix a small bug with the sensorshell controller which would create multiple instances instead of just one. I also added some simple controls for manually shutting the sensorshell control down without reset the server. Theoretically this would be useful if the sensorshell needs to be updated while the server is running. Any data received during that time is of course lost though.

I also extended the functionality of the subscription service. Team Foundation Server allows users to specify XPath expressions when subscribing to a service and I put in the necessary GUI to allow for that to work. This lead me to realize that I had not been validating any of my input fields. Since its so easy to validate input in ASP.NET I added some validation controls and used the C# XPath library with these controls to check whether the XPath expression was correct. It was pretty easy to do but it really made things alot more robust.

Lastly I added some extra settings to allow users to easily specify default settings before the server loads. The original settings are loaded from a file and then additional settings can be loaded from the VSTFS database if they wish. An example settings file. If a setting is missing or corrupt a default setting will be created upon server load.

Lastly. I was also able to figure out how to link external cs files into the Unit Test project so that I did not need to copy them.

Turn-on, tune-in, and get ready to update!

…and were Live!

Ok, so it took me all day but I finally did it. As promised:

I know what your thinking. Only 69 tests, thats really not too many. I know, I know, but believe it or not it took me a long time to get those tests working 100%. I spent the better part of today refactoring and reworking the tests to try and get them all to pass. They are pretty basic but amazingly enough they enabled me to find some pretty big bugs which I had subsequently fixed.

I’ve been learning alot about C# and the whole visual studio team system suite lately. Stuff like proper assembly usage and common coding practices that I never really needed to delve into when using c# before. Though I must admit that some things that would take experienced Microsoft programmer seconds to do sometimes took minutes or hours since I had to look through various documentation to find the right line to add to fix everything.

The Unit Test Hour:

Unit testing is fun! Trying to break your own software is just cool. After spending countless hours trying to perfect a specific function or class its nice to just let loose and try to break it, ON PURPOSE! It’s also very satisfying to see all the little green (and red) icons light up after hours of black and blue coding; pun intended. I have no problem breaking my own code, it also usually gives me a good idea of what to focus on next.

After hours of trying various approaches to Unit Testing Web Code and coding some of the more basic unit tests for other classes I eventually realized that I could use the JAXB approach. Here’s what I did. I used the template classes that are used to de-serialize the incoming SOAP XML from TFS and instead created a method which, when presented with the appropriately filled class serialized the data. I then passed this “sample data” to the methods which handle the incoming TFS SOAP XML and added code to each “notify” method to return true or false based on whether the sensor could send data or not. Very JAXBish. The only problem with this method though was that the Unit Tests were located in another project and I couldn’t find a way to access the classes I needed without copying them over to the Unit Test project. VERY BAD PRACTICE. But for now it seems to work. If anyone can suggest a better way of referencing classes from another project under the same solution please, please, please email me. I think it has something to do with assemblies.

Another thing to note. You should always make sure to develop your unit tests as self sufficient tests. Assume no order to which the tests are run. I came across this problem when I was writing my tests. Early on I found that some of the tests which queried the database would fail when run the first time. On a second run however, some would fail and some would pass. Very strange. On investigation I found that this occurred because the first time they ran they modified the entries (or the database itself!) which then created a different test context the second time around. I think I’ve gotten it a bit more balanced now. I create an entirely new database for testing. After I am done testing I simply delete the newly created database. It’s slow and cumbersome but it works really well and I don’t think people will be testing on a production server so I think its a good compromise for developers.

In the know: Namespaces

Useful little things. At first I didn’t see a need for them but they really help to separate the structure of classes and code. I am currently trying to add namespaces to my classes so that I can see a logical grouping of code that belongs together in the object browser.

Documentation with Matt in the Morning:

I’m finding alot of problems that I am having is with documentation. Surprise, surprise. Though I am not quite having the problems you would think. Sure there is a lack of good documentation on lots of things but I think that is to be expected. The real problem I am having has to do with legacy documentation. I’ve been reading alot of documentation which ends up being the wrong documentation for the current version of the software I am using. Whether its the hackystat version 7.0 docs or the Visual Studio 2005/2008 documentation, people tend to eithier incorrectly label or provide very few labels as to which version of software the documentation pertains to. Then there is the problem of old documentation when the current archive is moved to a new location, updated, and the developers fail to remove the old documentation. I think a critically overlooked problem with documentation for developers is that they must keep all documentation for each version clearly labeled, sorted, separated and in one place. This is probably one of the reasons Wikipedia does so well. Its all organized in one place and its extremely difficult to find old versions of wiki pages. Sometimes its actually a good idea to either update the documents or simply delete them entirely to force people to find the new ones. Archiving like pack-rats does not a good developer make.

…and he rode into the courtyard, at sunset, with only seconds to spare: GUI

Well its not that good but I spent some time this week redeveloping the structure of the GUI.

Its a bit bland and has many css errors but it looks better than before. I will probably change it a few more times before the end of the project. Suggestions are always welcome.

Whats next

Now I plan to do some more advanced things. Besides better unit tests I am going to delve into the Team Fortress, I mean Foundation, Server and see what data I can pull in addition to what is given when an event occurs. I should be able to pull at least a little more data. I was also thinking of comparing changesets from source control when a new changeset is checked in to give a better idea of what exactly developers are checking in.

B.T.W. This blog GUI really makes my post look longer than it really is.

B.T.W.x2 Anyone notice how most programs are adding in instant spelling support with little wavy red lines right under our noes. I just used TortiseSVN today and when I spelt something wrong it suggested the correct spelling.

House keeping

Today I did some very overdue house-cleaning as well as setup the unit-test infrastructure. First off I reinstalled VMware with a new version in hopes it would provide better speed. It did. Next I installed (finished installing) Visual Studio Team Systems 2008 and its corresponding Team Center add-in so I could access the Team Foundation Server and the project it was housing. I then set to work creating a new c# test project which will be used to create unit tests for the system.

Then the trouble began. Source Control is a fickle thing, and it seems as though all of the updates, renaming, moving, offline deleting and general mucking about with forces well beyond my comprehension had confused the Team Foundation Source Control Server so badly that it wouldnt add or delete files I had changed. I did the only thing I could do which was to backup the hard-files I had been working on and delete the project. (Don’t worry all, I have a second version on SVN at Main Repository ). I then created an entirely new Source Control project, added all the old files, converted the project to .NET 3.5, created a new unit test sub-project and then commited everything to both the Main Repository and the Backup. Now I know what not to do in the future.

I feel as though I have been lacking with the substantial updates and pictures as of late. Therefore I hope to post a huge update this Friday.

The Good, The Bad, The …

The week started off very well. I was able to finish off implementing sensors for all of the Event Services. Unfortunately, I just learned that I may have done things a tad incorrectly as I was using the specifications for Hackystat version 7 instead of version 8. Not a big problem as it should only require a small change to correct. I was also able to fix smaller connection issues which resulted in the data being kept offline even when it should have been sent to the Hackystat server. It seems I had been overriding the “Owner” field which is used to validate that the data is being sent from a user that exists in the hackystat database. Replacing this with the owner who sent the data from the VSTFS database made Hackystat get confused about who was sending what.

I’ve been encountering some other strange connection issues. Some data seems to be getting lost somewhere. If I am offline the data correctly gets stored in niftly little xml files by the sensorshell and when I next go online they get sent out fairly quickly. Though when I am already online some of the data does not always get transmitted through.

My hypothesis:

a. Data is being lost because its formated incorrectly. Easy to test, easy to fix.

b. Data is being lost because of the current server setup. Not so easy to fix.

I suspect it may be more of B than A. It may be that because the data must fly through the sensorshell, then through the virtual machines “virtual network card” to the host computers real network card to the network then into the server (which is running quite slow), it may be that the server just isn’t getting the data and since my side (client) technically sent the data its not getting stored offline either. In any case I now have a critical priority ticket to fix the problem. Thats goal number 1 for next week.

A second problem I had was that my version of Visual Studio didn’t have the test suite tools add-in installed on it which meant I couldn’t write any proper unit tests this week. Luckily I obtained the Visual Studio 2008 Team Suite through MSDNAA and have installed it so that I can now start writing some tests (YAY!).

Another piece of good news is that Greg Wilson has pointed me to Jean-Luc David who has written two books on TFS (Team foundation Server) and VSTS (Visual Studio Team Systems) and has graciously offered to answer some of my more pressing questions about them. Hopefully this means next week I will be able to ramp up really quickly and get version 1.0 of the sensor out the door.

By the by: I plan to have version 3.0 done by summers end. (everyone knows that 3.0 is always the best version 😉 )

IT LIVES!

Today was a pretty big day. I spent part of the day (lets say the morning) trying to find out what caused some of the more esoteric events that people can subscribe to with VSTFS. These events include:

• Any of the Node******* events
• Any of the Identity******* events
• The common structure change event
• Data change event

Now that I have a good idea of what these events are and when they occur I was able to simulate their occurrences and make the appropriate additions to the code to allow for them to be handled by the web service. This means that my web service now crudely supports all 17 built-in VSTFS events and sends the appropriate information to the local UofT Hackystat server (I plan to test this later in the week… but theoretically it should work :p ).

As I am sure the names above are as confusing to you as they were to me the first time I saw them I am putting up documentation about each event and how you can trigger them on my wiki. I don’t have them all up yet but I plan to by weeks end. If your really interested in the project I highly suggest taking a look. The descriptions for each are short and each includes a small screenshot indicating how I went about triggering the event.

My plan for the rest of the week is now three-fold. I must begin to write-up the wiki documentation on what I have done so far, I need to create tests for each of the events that can be sent to the web service and I must start investigating extracting more information than is provided from the web service from VSTFS.

P.S. Anyone remember what step I am supposed to be on at this point in the project?