In OSD600 I completed three iterations, 0.1-0.3. Now in OSD700, I need to do 0.4-1.0.

This post will outline what I did for 0.4.

I’ve started by taking Firefox bug 677122 which was labeled as “not quite as easy”. I am always up for a challenge. For the tl;dr check my diff.

I will talk about what I did right, what I did wrong, and what I learned.

Three main things I did in this release are:

Created a working Firefox build on Windows. Which was not quite as easy because of bug 718541. I ended up narrowing what was needed to get a working build, and adding my findings to the ticket. Now being able to reproduce this problem, in hopes that it can be used to track down the exact cause. The ticket is now fixed.

Got a working developer environment working on Windows with Visual Studio 2010. This took more time than I wanted, as I am so used to web development where things like firebug work right out of the box. In other courses and projects, the code base is small enough that I don’t need any real developer tools, except my mind.

The real work was done in bug 677122. I had three attempts on this, and will describe each one, and why the first two failed and last succeeded.

First attempt was to wire the go to anchor code to process media fragment code. This failed because process media fragment should not be called out of context. It was originally a protected function, so that there should of been a red flag that this was going to fail. I moved it to public, then in the anchor code I would get a reference to the media element, and call ProcessMediaFragment, which would crash when trying to grab the mLoadingSrc. I suspect this is due to the way the media element lives in the media document. I loaded up firebug and actually looked at it, and it did not even have a src attribute attached to it. Probably why it failed, as it was not completely what I wanted. I did manage to parse the fragment code so I knew the difference between a fragment and an anchor, which would be crucial in the third attempt.

Second attempt was to try to get around the loading src, by doing this at a different time, or by calling a different function. Both attempts failed for similar reasons. I don’t think the timing was even the problem, but the reference to the element itself. A media document’s video element is not complete, and should not be used as such. It is a different beast. I tried things like, LoadWithChannel and resetToURI which both failed because they would access an element that was not what they expected it to be. Again, it doesn’t have a src attribute like most media elements. I also tried to do the work in the InternalLoad function, before the anchor code. This failed for the same reasons, but did get me choking off the GoToAnchor code, which helped me find the next part.

Third attempt was actually putting together two elements from the failed attempts. I moved the check for anchor or fragment to the InternalLoad, where I choked off the anchor code. It was as simple as that. Check for fragment and choke of the anchor. Once done, let the rest of the code to continue on its way. I have created a patch for this and submitted it for feedback in the ticket.

The biggest lesson I learned was a soft lesson. I should of probably asked for help between Monday’s post and Wednesday’s post. In retrospect, reading those posts now, I can see the naivety in my attempts. I was so positive about some of my solutions, and I can see it in my words. This will get better when I get more comfortable in the community, as I have been here before.

Some smaller lessons were more technical things about Firefox code. Like how to get a reference, cast it, string management, etc. These are things specific to Firefox code, and not really C++.

Warning, some of the links to Firefox source code will bitrot as the code changes and the lines where things live changes. Not sure of a better way to link to the source code.

Thanks for reading

What I have accomplished is getting a reference to the video element, inside the video document, at the time the user clicks enter while selecting the address bar that has some media fragment data. Test video link.

I also accomplished exposing the ProcessMediaFragmentURI function to this scope. This was my initial goal, and what I hoped could be the finish line. That function should be doing all I need and This is a huge step, but there is one small problem. That function crashes if the video has already been loaded.

I am still wrapping my head around why this happens. What I do know:

The existing code only calls ProcessMediaFragmentURI inside the MetaDataLoaded. So currently, this function is only ever called during a load.

So, from inside ProcessMediaFragmentURI, GetCurrentSpec is being called. It hits line 2880 and checks if something called mLoadingSrc is true. If the video is already loaded, this enters an object, and starts the process of a termination of the browser.

I have options. I can either investigate this termination, possible bug, and make the ProcessMediaFragment function work on loaded and playing videos. Or, I can figuring out how to properly call LoadWithChannel, which will call ProcessMediaFragment inside a loading context. The later is what I am currently building, as it seem like the path of least resistance. But, I suspect it may not be the best solution in the end.

The problem in the bug is that a media fragment of a video document will not update changes unless you do a refresh of the page.

The media fragment parsing happens once the video’s meta data is loaded, which makes sense. So it is obvious why the fragment doesn’t get re parsed without a refresh. Something I did notice though, in the anchor code a string with the media fragment data does get passed in, but I have not found it using this data for anything. Seems like a waste, and to me it would make sense to just do all media fragment parsing then. One point, instead of two. I cannot simply refresh the video when it is already loaded, nor can I set the video’s playback position if it did not get loaded.

My options are to hook into the anchor code, parse for “t=10,100″, if we are a video doc, and video is loaded, call media fragment parser, if the video is not loaded, setup an event of some sort. One path to all “#” at end of URL, two ways to parse it (media fragment or anchor), loaded or not. Or, keep things the way they are, and just do half of what I said above; only update the fragment in the anchor parse if the video is already loaded.

One of the first things I found was that most of my previous techniques would not fly in a code base this big. It is just not effective to insert a log message and recompile. It will take too long to compile, and the data you log will just be too little too late. This technique may still be viable, but just in a limited space. I also have a technique for seeing what code does, by commenting it out, and running the program to see what the program no longer does. This is also not viable because of the size and compile time.

The simple technique of reading code will also only get you so far. This code is just too big; you need stack traces, breakpoints, and the ability to see the value of every variable in scope.

I tried two things to get the tools I needed to do this.

First I tried GDB (GNU Debugger). I am currently using Windows, so this was a bit of a challenge, and I got it kinda working. But, I still don’t think it was enough, because I could only get break points and stack traces; I did not get the ability to see variable values.

Second was to try Visual Studio. This was actually much easier than I expected. I used this guide. I was happy to finally find some instructions that were not fragmented, but up to date and on my system and versions that I am running. So it all went smoothly, which I did not expect. This allows me to set a break point right in the source, run the program, and it will stop execution at those points and let me explore the scope and stack trace. This, along with Mozilla’s cross reference (MXR) will allow me to hack and debug Firefox.

I now have a really nice workflow.

I figure I need to find the code that initiates the parser for the URI and also find the code that should refresh the URI, and re initiate the parser. I notice doing something similar with a URL anchor, changing it, then hitting enter updates the page. So I should also find this code, and see how that happens.

The very first thing I did was went to view some source code. I went to Mozilla cross reference. This is an online search and reference of the Firefox source code. I searched for “media fragment” and found nsMediaFragmentURIParser.cpp, and from there, found nsHTMLMediaElement.cpp. Some really exciting code in the HTMLMediaElement code is ProcessMediaFragmentURI. This block of code calls the parser, and sets the video’s time based on the values returned from the parser. So now, I need to find the places that should also be doing this.

I think though, this is as far as I can go with simply reading source code like a novel. I have some code that looks interesting, and is a good place to start. It is time to start breaking the code. By that, I mean remove, change, or add some code in areas that look interesting, then compile the code, run it, and see if my changed code runs when I suspect it will run. To do this, I need to get the source, and be able to compile it.

I have done this before, I even have blog posts on doing this… but it was ages ago, but things have changed, and I need an update anyway, so I think I will start freshfrom the online build instructions. I am doing this on a Windows 7, so that may add an extra level of complexity.

I need Visual Studio of sorts. I have Visual Studio 2010. I also need to download the Mozilla build tools for Windows. Once downloaded and installed, I run start-msvc10.bat, with the 10 being for that I use VS 2010. If I had VS 2008 I would of ran start-msvc8.bat, etc. All commands are run inside this shell.

Next I download the source code by running “hg clone http://hg.mozilla.org/mozilla-central”. My first attempt I got “abort: stream ended unexpectedly (got 98688 bytes, expected 125068)”. Second time, no errors, took forever.

Next is to build it. I do this by changing into the new directory mozilla-central, which was created when I did the last command. Then, I run “make -f client.mk” and wait. On my Windows laptop this took 30+ minutes, but it seemed to finish without a hitch. I may consider going through tips for making builds faster, but for now, I will confirm that it worked. After the build is done, to confirm that it worked, I ran “obj-i686-pc-mingw32/dist/bin/firefox.exe”.

That concludes my adventures in building Firefox and exploring the source code. Next time, I will get into hacking and debugging. Then maybe making a diff and submitting a patch!

A tl;dr demo.

The idea is to execute the goggles between a time range, on an element defined by popcorn’s data. In the case of the above demo, I used the video element for testing purposes.

Right now, it is in a demoing state, demoing the use of the goggles at a time range. It is not really in a practical usable state, as you cannot close the goggles, nor can you control the video as they are up; quite frustrating.

I am going to allow popcorn’s data decide what to do with the timing. It can either leave the video playing, pause the video for a set number of seconds, or simply leave the video paused until the user closes and hits play again.

I also need to re add some animation that I avoided for sake of getting it working. Doing this will be good practice for me, because I’ve never been much of a front end guy, but I have the artistic abilities that I just need to dust off.

The hardest part of this was using HTML5′s postMessage. I have never used it before, and was stuck for longer than I would of liked, but long enough to build character. I was attempting to send a postMessage to an iframe before the iframe was done loading, thus the iframe’s event listener was not yet defined, thus my postMessage fell on deaf ears.

First, It turns out media fragment is already done, which is exciting because now we can use it in Popcorn.

Second, three bugs were blocked by it, which can now be done: [498253 677121 677122].

Reading bug 498253, and it seems as of second quarter of last year, there were still on going spec changes.

677121 still interests me.

And, 677122 seems straight forward enough. A bug where it seems it is caching the media fragment start time, and a refresh is required to update the time. The steps to reproduce right from the bug:

1) Open http://www.bluishcoder.co.nz/mediafrag/AudioAPI.webm#t=50,100
2) Change the fragment portion to be “#t=30.100″ and press enter

Expected Results:

1) Start and End fragment of the video would be the new position of 30 and 100 respectively.

Actual Results:

1) The video does not change the start/end fragments. They stay at 50 and 100.

This one is still a good candidate.

I’m currently working at CDOT with Dave still, which means I will still be working on Popcorn and Popcorn-Maker. But, for his class, I will need to do more work for him, but it cannot be CDOT work. It makes sense to me that I find Firefox bugs involving video spec, that way I can leverage these new tools into popcorn and popcorn-maker use.

So let’s look at bugzilla.

I made a general search under “media”. I wasn’t expecting anything to jump out, just wanted to see what I was dealing with. But, something did jump out at me.

Now, this ticket might not be the starting point for my work, but it is definitely something I am interested in.

The general interest is media fragments. Which the above bug relates to, but does not mean I am not doing the above bug. More research is required.

I also have interest in general media spec bugs, and I’ll look for more to have some concrete bugs tomorrow night.

A bookmarklet is JavaScript that runs via the address bar. So, you can fill a link element in a webpage to instead run code. Click the link, runs the code. You can also drag said link into your bookmarks for later, and continual use.

What I’ve done is created a bookmarklet that lets you view and edit any popcorn.js page real time.

There are a few caveats. 1. It cannot be done on any version of Popcorn before 0.4, because I need access to Popcorn.instances, but that doesn’t matter because it also needs p.media, which was introduced in 0.5. I also ended up using Popcorn.getScript, which is another dependency, but getScript is introduced before 0.5, so if we have instances and media, we’ll have getScript. 2. We need plugins with complete manifests. Manifests are like meta-data for plugins. It tells me what to expect, thus what I can enter into the form fields. Without a manifest, it is read only Popcorn. 3. Demos made from exec alone will not work. Exec isn’t really editable without writing JavaScript anyway.

Now that the academic parts of the post are out of the way, I shall ramble about my thoughts on the project. This thing, to me, is a good example of a hack. I took parts of code from one thing, and remixed them together to do something different. I had little regard for how it looked, just that it worked. So, I guess it is more of a proof of concept. I really want to take it to the next level, though. Give it an export of some sort, and more closely tie it in to Popcorn-Maker’s build tools. Other-wise, this is essentially a fork of Popcorn-Maker. Over time, it will be too different, and become a separate entity that needs to be maintained separately. In order to avoid this, we would have to do some changes to Popcorn-Maker. It has some flaws, things I had to hack around. One of the good things was Popcorn-Makers event system. Made it easy to tie in my new functionality without having to worry about what was happening in other parts. Something interesting I noticed doing this, Popcron-Maker is close to 40k lines of code. This includes HTML, CSS and remote scripts like JQuery. Still though, that’s a lot of lines.

A few good pages to try it on are: Popcorn homepage, The demo from this fantastic tutorial on Popcorn itself, and the initial semantic video demo from way back in 0.1 of popcorn. It’ll work on others, but there is code in the wild it will not work on, yet. I’m going to deploy the “fix it for one demo at a time” technique I learned doing work in Processing.js. It’s great to have user generated content to test off of!

Now for an abrupt end to this post.

Yesterday Toronto time we released Popcorn.js 1.0 in London time.. or yesterday, I’m not sure, but anyway, it was super cool. We released with new docs made by David Seifried, a Mozilla based website for what Mozilla is doing with Popcorn, and a shiny new build tool, built by Chris De Cairos.

I’ve been spending the weekend in London at the Mozilla Festival. Been getting way more Popcorn attention than I expected or imagined! Been interesting hearing impressions and educating people on Popcorn.

I also spent some time at the festival hacking up a Popcorn.js Universal Subtitles parser, and demo. I have one work in here before. Back in 0.1 I did a internal parser for Universal Subtitles, and for a beta version of Popcorn-Maker, I implemented a parser for Universal Subtitles as well. This one though, should be the final, as it’s not longer living inside something, but lives on it’s own in modular space. I also decided to take some time to make it look nice with Brian Chirls’ pop plugin! Check it out!