The limitations of iBooks when playing audio files in interactive EPUB3 format – through the developers’ eyes

Posted by | | Company life | No Comments on The limitations of iBooks when playing audio files in interactive EPUB3 format – through the developers’ eyes

In June, 2012 our company published our first interactive book for iBooks capable iOS devices. Taking the idea further, in December the same year we created and published the first episode of the Tales from the Playroom series, entitled First Colours megérkezik in Hungarian and The Arrival of Erste Farben in English. In the course of the development we encountered a number of limitations that appeared in iBooks, but not in the simple WebKit-based browser.

Interactive EPUB3

An interactive EPUB3 format book is technically series of HTML5 pages, equipped with all the weapons of HTML5. This is what we started out from as well.

In the course of the development we mainly had to rely on our own experiences since documentation in this field is scarcely available.

In the course of the development we mainly had to rely on our own experiences since documentation in this field is scarcely available.
There are development tools, with which it is possible to completely build an interactive EPUB3 format book, but these did not give us sufficient freedom in creating the interactions, animations we had imagined, thus we had to build the books that fulfilled our needs from scratch. We achieved this using HTML5, CSS3 and JavaScript.
When using these, we encountered a number of limitations, anomalies and defects which the iBook is struggling with. The WebKit-based browsers, the latest versions of Safari and Chrome provided the basis for comparison.

apple_edu_ibooks_video_heading

iBooks vs HTML5 audio

With HTML5 we are able to use audio and video elements in the interactive books created in EPUB3 format. So far, everything is all right. We came across a fault in connection with playing sound effects of dynamically varying numbers at the same time, which is annoying in terms of performance.

Everything is perfect if we place an element in the XHTML, and try to play it. However, if we do not know in advance how many times we wish to play the given sound parallelly, problems occur.

It does not help if we place the given audio file the following way:

<audio id=”foo” preload=”auto” src=”audio/foo.m4a”><span>foo</span></ audio>

the iBooks “decides” that it does not have to place this globally in the cache. By “globally” we mean that for instance if we make the browser load an image in the DOM (irrespectively of whether it actually appears or not), then when placing a new element in the DOM with the same URL, it takes the image from the cache so it appears immediately.

This would seem the logical action, but this is not included in the HTML5 specification.

As a result, when we wanted bubbles to emit a sound when they are are burst in our solution, it failed to work in iBooks.

It worked beautifully in the Chrome/Safari browsers, and it was enjoyable and did not slow down. It was useless in iBooks, although we added the audio element to the DOM when the bubble appeared, thus it would have had enough time to load the sound, but even the animation got stuck (which should be running on a separate thread).
We based the code section playing the sound on the code in the FixedLayoutExample that is freely downloadable from iTunesConnect.

Screen Shot 2013-06-06 at 15.19.16

The solution we finally got can be tried by clicking here in a browser, and here it is available in EPUB3 format.

The lesson regarding development in iBooks and the experienced faults is definitely the fact that it is essential to test it on the target devices, and resource management is also vital in order to achieve a better user experience.


No Comments

Leave a comment