What I have accomplished is getting a reference to the video element, inside the video document, at the time the user clicks enter while selecting the address bar that has some media fragment data. Test video link.

I also accomplished exposing the ProcessMediaFragmentURI function to this scope. This was my initial goal, and what I hoped could be the finish line. That function should be doing all I need and This is a huge step, but there is one small problem. That function crashes if the video has already been loaded.

I am still wrapping my head around why this happens. What I do know:

The existing code only calls ProcessMediaFragmentURI inside the MetaDataLoaded. So currently, this function is only ever called during a load.

So, from inside ProcessMediaFragmentURI, GetCurrentSpec is being called. It hits line 2880 and checks if something called mLoadingSrc is true. If the video is already loaded, this enters an object, and starts the process of a termination of the browser.

I have options. I can either investigate this termination, possible bug, and make the ProcessMediaFragment function work on loaded and playing videos. Or, I can figuring out how to properly call LoadWithChannel, which will call ProcessMediaFragment inside a loading context. The later is what I am currently building, as it seem like the path of least resistance. But, I suspect it may not be the best solution in the end.

The problem in the bug is that a media fragment of a video document will not update changes unless you do a refresh of the page.

The media fragment parsing happens once the video’s meta data is loaded, which makes sense. So it is obvious why the fragment doesn’t get re parsed without a refresh. Something I did notice though, in the anchor code a string with the media fragment data does get passed in, but I have not found it using this data for anything. Seems like a waste, and to me it would make sense to just do all media fragment parsing then. One point, instead of two. I cannot simply refresh the video when it is already loaded, nor can I set the video’s playback position if it did not get loaded.

My options are to hook into the anchor code, parse for “t=10,100″, if we are a video doc, and video is loaded, call media fragment parser, if the video is not loaded, setup an event of some sort. One path to all “#” at end of URL, two ways to parse it (media fragment or anchor), loaded or not. Or, keep things the way they are, and just do half of what I said above; only update the fragment in the anchor parse if the video is already loaded.

One of the first things I found was that most of my previous techniques would not fly in a code base this big. It is just not effective to insert a log message and recompile. It will take too long to compile, and the data you log will just be too little too late. This technique may still be viable, but just in a limited space. I also have a technique for seeing what code does, by commenting it out, and running the program to see what the program no longer does. This is also not viable because of the size and compile time.

The simple technique of reading code will also only get you so far. This code is just too big; you need stack traces, breakpoints, and the ability to see the value of every variable in scope.

I tried two things to get the tools I needed to do this.

First I tried GDB (GNU Debugger). I am currently using Windows, so this was a bit of a challenge, and I got it kinda working. But, I still don’t think it was enough, because I could only get break points and stack traces; I did not get the ability to see variable values.

Second was to try Visual Studio. This was actually much easier than I expected. I used this guide. I was happy to finally find some instructions that were not fragmented, but up to date and on my system and versions that I am running. So it all went smoothly, which I did not expect. This allows me to set a break point right in the source, run the program, and it will stop execution at those points and let me explore the scope and stack trace. This, along with Mozilla’s cross reference (MXR) will allow me to hack and debug Firefox.

I now have a really nice workflow.

I figure I need to find the code that initiates the parser for the URI and also find the code that should refresh the URI, and re initiate the parser. I notice doing something similar with a URL anchor, changing it, then hitting enter updates the page. So I should also find this code, and see how that happens.

The very first thing I did was went to view some source code. I went to Mozilla cross reference. This is an online search and reference of the Firefox source code. I searched for “media fragment” and found nsMediaFragmentURIParser.cpp, and from there, found nsHTMLMediaElement.cpp. Some really exciting code in the HTMLMediaElement code is ProcessMediaFragmentURI. This block of code calls the parser, and sets the video’s time based on the values returned from the parser. So now, I need to find the places that should also be doing this.

I think though, this is as far as I can go with simply reading source code like a novel. I have some code that looks interesting, and is a good place to start. It is time to start breaking the code. By that, I mean remove, change, or add some code in areas that look interesting, then compile the code, run it, and see if my changed code runs when I suspect it will run. To do this, I need to get the source, and be able to compile it.

I have done this before, I even have blog posts on doing this… but it was ages ago, but things have changed, and I need an update anyway, so I think I will start freshfrom the online build instructions. I am doing this on a Windows 7, so that may add an extra level of complexity.

I need Visual Studio of sorts. I have Visual Studio 2010. I also need to download the Mozilla build tools for Windows. Once downloaded and installed, I run start-msvc10.bat, with the 10 being for that I use VS 2010. If I had VS 2008 I would of ran start-msvc8.bat, etc. All commands are run inside this shell.

Next I download the source code by running “hg clone http://hg.mozilla.org/mozilla-central”. My first attempt I got “abort: stream ended unexpectedly (got 98688 bytes, expected 125068)”. Second time, no errors, took forever.

Next is to build it. I do this by changing into the new directory mozilla-central, which was created when I did the last command. Then, I run “make -f client.mk” and wait. On my Windows laptop this took 30+ minutes, but it seemed to finish without a hitch. I may consider going through tips for making builds faster, but for now, I will confirm that it worked. After the build is done, to confirm that it worked, I ran “obj-i686-pc-mingw32/dist/bin/firefox.exe”.

That concludes my adventures in building Firefox and exploring the source code. Next time, I will get into hacking and debugging. Then maybe making a diff and submitting a patch!

A tl;dr demo.

The idea is to execute the goggles between a time range, on an element defined by popcorn’s data. In the case of the above demo, I used the video element for testing purposes.

Right now, it is in a demoing state, demoing the use of the goggles at a time range. It is not really in a practical usable state, as you cannot close the goggles, nor can you control the video as they are up; quite frustrating.

I am going to allow popcorn’s data decide what to do with the timing. It can either leave the video playing, pause the video for a set number of seconds, or simply leave the video paused until the user closes and hits play again.

I also need to re add some animation that I avoided for sake of getting it working. Doing this will be good practice for me, because I’ve never been much of a front end guy, but I have the artistic abilities that I just need to dust off.

The hardest part of this was using HTML5′s postMessage. I have never used it before, and was stuck for longer than I would of liked, but long enough to build character. I was attempting to send a postMessage to an iframe before the iframe was done loading, thus the iframe’s event listener was not yet defined, thus my postMessage fell on deaf ears.

First, It turns out media fragment is already done, which is exciting because now we can use it in Popcorn.

Second, three bugs were blocked by it, which can now be done: [498253 677121 677122].

Reading bug 498253, and it seems as of second quarter of last year, there were still on going spec changes.

677121 still interests me.

And, 677122 seems straight forward enough. A bug where it seems it is caching the media fragment start time, and a refresh is required to update the time. The steps to reproduce right from the bug:

1) Open http://www.bluishcoder.co.nz/mediafrag/AudioAPI.webm#t=50,100
2) Change the fragment portion to be “#t=30.100″ and press enter

Expected Results:

1) Start and End fragment of the video would be the new position of 30 and 100 respectively.

Actual Results:

1) The video does not change the start/end fragments. They stay at 50 and 100.

This one is still a good candidate.

I’m currently working at CDOT with Dave still, which means I will still be working on Popcorn and Popcorn-Maker. But, for his class, I will need to do more work for him, but it cannot be CDOT work. It makes sense to me that I find Firefox bugs involving video spec, that way I can leverage these new tools into popcorn and popcorn-maker use.

So let’s look at bugzilla.

I made a general search under “media”. I wasn’t expecting anything to jump out, just wanted to see what I was dealing with. But, something did jump out at me.

Now, this ticket might not be the starting point for my work, but it is definitely something I am interested in.

The general interest is media fragments. Which the above bug relates to, but does not mean I am not doing the above bug. More research is required.

I also have interest in general media spec bugs, and I’ll look for more to have some concrete bugs tomorrow night.

A bookmarklet is JavaScript that runs via the address bar. So, you can fill a link element in a webpage to instead run code. Click the link, runs the code. You can also drag said link into your bookmarks for later, and continual use.

What I’ve done is created a bookmarklet that lets you view and edit any popcorn.js page real time.

There are a few caveats. 1. It cannot be done on any version of Popcorn before 0.4, because I need access to Popcorn.instances, but that doesn’t matter because it also needs p.media, which was introduced in 0.5. I also ended up using Popcorn.getScript, which is another dependency, but getScript is introduced before 0.5, so if we have instances and media, we’ll have getScript. 2. We need plugins with complete manifests. Manifests are like meta-data for plugins. It tells me what to expect, thus what I can enter into the form fields. Without a manifest, it is read only Popcorn. 3. Demos made from exec alone will not work. Exec isn’t really editable without writing JavaScript anyway.

Now that the academic parts of the post are out of the way, I shall ramble about my thoughts on the project. This thing, to me, is a good example of a hack. I took parts of code from one thing, and remixed them together to do something different. I had little regard for how it looked, just that it worked. So, I guess it is more of a proof of concept. I really want to take it to the next level, though. Give it an export of some sort, and more closely tie it in to Popcorn-Maker’s build tools. Other-wise, this is essentially a fork of Popcorn-Maker. Over time, it will be too different, and become a separate entity that needs to be maintained separately. In order to avoid this, we would have to do some changes to Popcorn-Maker. It has some flaws, things I had to hack around. One of the good things was Popcorn-Makers event system. Made it easy to tie in my new functionality without having to worry about what was happening in other parts. Something interesting I noticed doing this, Popcron-Maker is close to 40k lines of code. This includes HTML, CSS and remote scripts like JQuery. Still though, that’s a lot of lines.

A few good pages to try it on are: Popcorn homepage, The demo from this fantastic tutorial on Popcorn itself, and the initial semantic video demo from way back in 0.1 of popcorn. It’ll work on others, but there is code in the wild it will not work on, yet. I’m going to deploy the “fix it for one demo at a time” technique I learned doing work in Processing.js. It’s great to have user generated content to test off of!

Now for an abrupt end to this post.

Yesterday Toronto time we released Popcorn.js 1.0 in London time.. or yesterday, I’m not sure, but anyway, it was super cool. We released with new docs made by David Seifried, a Mozilla based website for what Mozilla is doing with Popcorn, and a shiny new build tool, built by Chris De Cairos.

I’ve been spending the weekend in London at the Mozilla Festival. Been getting way more Popcorn attention than I expected or imagined! Been interesting hearing impressions and educating people on Popcorn.

I also spent some time at the festival hacking up a Popcorn.js Universal Subtitles parser, and demo. I have one work in here before. Back in 0.1 I did a internal parser for Universal Subtitles, and for a beta version of Popcorn-Maker, I implemented a parser for Universal Subtitles as well. This one though, should be the final, as it’s not longer living inside something, but lives on it’s own in modular space. I also decided to take some time to make it look nice with Brian Chirls’ pop plugin! Check it out!

Wired has an article about the event, so check it out.

The actual event went down like this. Breakfast and mingling for about an hour, while everyone arrives. Then we all had a meet and greet. By 10:00 (if my memory serves me) we broke off into groups. So there I was with a group of people I never met, a film I have never seen, and a clock ticking. By lunch we had specced out what we were going to do. This is always an interesting process. You have a developer that knows nothing about the project or how things should be done, and you have designers and filmmakers that don’t know what can and cannot be done or how to measure how long something will take to develop. The problem for developers is “the easiest thing isn’t always the coolest” and for filmmakers and designers “the time it takes to develop something has nothing to do with how cool it is”. The developers don’t know what should be solved, and the fimmakers don’t know what can be solved. I think the solution is for the filmmaker and designer is to start with a big vision, then to have a back and forth discussion. It really is about communication and an open mind. Developers don’t be scared if filmmakers want it all. Filmmakers, don’t be scared to want it all. Developers don’t be scared to say no. Filmmakers don’t be scared to be told no . Eventually you will come to something that is “quick to cool”. Then it is lunch time.

After lunch you have to start developing. At this point huge changes to the spec can be frustrating, but still, no harm in asking. You never know, that thing you want changed might actually be easier! so keep up the communication. The rest of the day I struggled with getting Facebook’s API to work, but I did. It was an unstyled, nonsensical, boring piece of what I do best .

The next day we got to putting in the design and data. These are two things you need to figure out in the first half of the first day, and if possible have the data ready before the end of the first day. I kept the design pretty clean, empty, but clean, so it was quite easy to plug in some css and add some images. This is where you need the designer again. So, designer during spec phase, not after, and designer during pull it all together phase, not before. Then it’s just a matter of testing and debugging, and it’s still ok to request changes, just understand it might not happen.

Communication was key. Filmmakers teaching developers, and developers teaching filmmakers. I feel like designers have already crossed this bridge.