Make an opinion, to throw it away

I’ve faced a number of challenges when it comes to software development since I started school at Seneca 3 years ago, and started work at CDOT one year ago. Syntax, APIs, style, communication, research, to name a few, all had one thing in common. They all had clear and final answers, I just had to find them. I am sure one could make the argument that they only had clear answers AFTER they were solved, but, while solving them, I still knew there was an ultimate, logical, correct destination. Once found, the answer worked for almost every case after that.

The challenge I find myself facing now, is that of my own opinion. No one opinion can be used for every problem, for every person.

I spend a lot of time, going through an iteration, or a solution to a problem, that I may not agree with, but I do anyway simply because I lack an opinion. I’m shown something, I don’t know what I think of it, so I do it, and figure I can make a better opinion of it after it’s done, which I never get a chance to, Because in order to find my opinion I will have to solve every problem once, for every solution, then, and only then, can I say what I truly think. A good example is I not-so-long-ago helped port browser mouse events in processing.js, to safari touch events on the iP(od/ad/hone). I lost the argument (I don’t think anyone was on my side) do I still think I was right? No. Do I think the solution that was used was right? No. There probably isn’t a right solution, just opinions.

The proposed solution is:

Make an opinion, then throw it away, and let logic determine what’s best. Logic must have, or is real proof, so I would need to “solve every problem once, for every solution”, only difference is, I’m able to throw it away for the next problem, because I know it won’t be viable for every problem.

My solution doesn’t really solve anything regarding opinions, I just changed my opinion of the problem.

There is an opinion for every problem, for every person. That is a lot of opinions… let’s move on.

Butter’s track editing user interface upgrade

So, moving forward with Popcorn, I’m directing my attention towards Popcorn’s authoring tool, Butterapp.

The two major things I’m working on:

First: The current iteration of Butter uses a canvas to edit, add, remove and display track events. I’m going to change that to be html elements using jquery draggable. I want it so if a track event is dragged outside of its track, a new track will be created. Also, if it is dragged between two tracks, a new track will be created, and inserted between the two tracks.

Second: The current iteration is using the plugin type as the term to sort the tracks on. I’m going to change it to be sorted by track location. A quick example of the idea, made by Bobby Richter.

I’m going to be attacking both of these at the same time. mainly, making the functionality of the second, but making it with HTML elements.

We’ll have a largely object oriented model, with three main components. A Tracks object, multiple Track objects, and multiple TrackEvent objects.

A Tracks object: (for multiple tracks). Only one of these will ever need to exist, so probably just an object literal with the needed functions/attributes. This would contain tools for adding/removing tracks. I also think the order these are displayed is important, which makes sense to be managed by the Tracks object, and not individual tracks. The importance of order is why I see things like splice, push and pop to be important, because I feel adding and removing tracks should be accessible via order or object, so long as you know one or the other, you can manage it. Would contain a addTrack, removeTrack, getTrack and length property.

A Track object: This would be for adding/removing TrackEvent objects. Order of TrackEvents probably wouldn’t matter to the Track object, as they would be ordered based on their times/size. Would contain a addTrackEvent, removeTrackEvent, name, getTrackEvent, and length.

A TrackEvent object: that is created with a plugin type “subtitle”, “footnote” etc. Which would also hold all the values of the track event. There would be added/removed to/from the Track object. This would probably just be a reference to the popcorn track Event object.

Next step is to find all references to the current track system in butter, and remove it. Then start adding pieces one by one, starting with making the plugins draggable. Second, Implement by objects and their APIs. Third, use my object APIs to link it all together. That is all for now… more later?

popcorn.js performance: It’s a science

Recently, I helped on a popcorn.js demo integration with soundcloud.

It was a lot of fun to work on, and it brought light down on a couple bugs. One stuck out, being a performance issue. In the demo we had the need to add thousands (5938) of tracks, which all needed to be processed, and it was slow.

Today and yesterday I worked with Rick Waldron to try to figure out a new solution to adding tracks.

Popcorn.js has two arrays of tracks, sorted by time to display and hide the track data. One array is for time to display, the other is time to hide. If you can trust the data is in order by time, hold a reference to the last track that was checked, and have some really good development tests, you can find the data of intrest much much faster, instead of loop through the whole array each iteration

The problem with this though, is data can be added to this array dynamically, at any time, which means data must be sorted each time data is added. As you could imagine, calling any form of a sort function 5938 times, it will begin to get slow. You can hold off on calling sort, but it’s almost impossible to know when to sort again, because of how dynamic things can be. We brainstormed ideas, including a sorted or not sorted flag, which would be checked against once inside the update code. This would mean a new variable is added, and that the update code would have one more task, in what is already time sensitive, optimized code. Not an ideal solution, but a viable idea.

Another idea was sort by default, with the option of the user to override the sort, and call an explicit sort function when they are finished adding data. This is also a viable idea, but the disadvantage is, it is not very user friendly. These things “should just work”.

Third options was, which was the one we ended up going with, to never sort, and make sure tracks are placed in the right location right off the bat. Thingk of it this way, the best way to clean your room, is to never make a mess of it in the first place, and that’s what we did.

What was really cool about all this, was the collaboration and teamwork that pulled this off.

Rick setup a jsperf url, that we could both modify and compare against together. We had a case using the old popcorn event code, and a case with code we could modify, setting a benchmark to beat. Then we both went to work trying ideas, comparing, and mergin ideas into one, until we came up with a jsperf we liked.

What these numbers mean: “we have a 23,891% performance improvement” which is over 9000.

About half way down the jsperf link, you’ll see a “run tests” button, which will then show you operations per second. Higher is better. Below that, you’ll see the cumulative test of everyone that has run these tests, their numbers, their browser, and how many times it was run. Higher is better, again.

Lighthouse logging me out and back in as someone else

The title is exactly what I thought was happening, despite how illogical it may sound, and also the first thing I desperately attempted to google, and as expected, google didn’t return anything. So I figured out why, and am now filling that gap in google with my solution.

So it seemed to be random. I would open a lighthouse tab, log in, go do some work, come back to the tab, hit refresh, and I would be logged in as someone else. That someone else was a work colleague, and was only logging me in as him. It happened 3 times before I figured something fishy was going on. At first I thought he just used my computer and forgot to log out.

We ended up tracking it down to the use of lighthouse API keys logging you in on any existing tabs of lighthouse. So the API keys were as good as a password.

This is one of those puzzles that simply seem impossible, but ends up being so obvious once you find it. Also, things are never as random as they seem.

Popcorn.js jsonp request for milestones

Today for the CDOT dashboard project, I finished up a php server to serve jsonp data for popcorn.sj’s milestone data inside lighthouse.

You can access it via JavaScript like so:

$.ajax({
    url: "http://scotland.proximity.on.ca/sdowne/sqlite/jsonp.php?service=lighthouse&callback=foo",
    dataType: "JSONP",
    type: "GET",
    success: function( data ) {
        console.log( data );
    }
});

With the data argument in the success function being your shiny new json data.

It is being tested and reviewed by Dave currently, who is the one that needed the jsonp data in the first place.

The obvious reason for this existing is to allow thing like json to be requested via server side scripting (JavaScript), and to my knowledge this does not exist yet for lighthouse. Another reason for this existing, is it reduces the number of calls to lighthouse. I was initially going to setup an internal timer, which would update the data served every half hour. I ended up thinking up a much much better solution. Instead, I would hold timestamps on the last time the data was updated, then, when someone requests that data, check if the timestamp is more than 30 minutes ago, if so, update via lighthouse json request, and set timestamp as the current time. This saves unneeded calls, so I ONLY update the server side data if a request is made, instead of a blind cron job. Also is much easier, and much less code.

The internals ended up storing the data on a very very simple sqlite layout. I am actually enjoying using sqlite via php very much. I just used php’s exec function to directly call and manipulate my sqlite database. No passwords, no connections, no remote.

For now I only serve popcorn’s milestone info, because there was no other way to get it, and I figured it would be a useful hello world test, but in the future we will add more to it, as needed.

I’ll be adding the source to github later today, but there is some setup that is required to get it working, that is why I setup a link on scotland.proximity.on.ca. Also, I figure it can be used by someone interested in this data for other projects.

First week back at CDOT: CDOT-DashBoard

This is my first week back at CDOT, and we a handful of new faces.

So far four of the new faces are rocking blogs: dseifried, Jon Buckley, Chris De Cairos and dperit.

Our first task is an internal CDOT status board to display status updates and overviews of our CDOT projects. Also, to get everyone familiar with git, github, irc, each other, JavaSciprt, and the work flow.

We broke it into three major components:

Widgets for displaying quantifiable data in a visual form. This data and visual display is highly unique per widget.

A framework for displaying and registering widgets in an overall frame.

Finally, a php file for caching all this remote data into one location all widgets can access the data from. This reduces calls to the various APIs for data that is shared by many widgets.

I threw up a quick example of a simple usecase of the framework, and an example milestone widget.