Introduction

 “Innovations like AR, iPads, Touchscreen, etc., can be much more if not underused by the entertainment industry” — Justine Bateman (DigitalDefynd, 2024).

Incorporating new technology into pre existing content can encourage engagement from younger generations and bring life to the content. Using augmented reality as a tool for learning in the topic of reading could encourage children through interactivity and technology. As stated in the quote above, it is the opinion of some in the industry that AR is underused by entertainment; this project will focus on testing the useability of implemented AR technology on existing content in the topic of children’s books.

The following is the major milestones to be completed for this project, as explained within the proposal.

The content for this project will consist of a portfolio (Introduction, idea generation and research, project management, software proficiency, ethics and values, forward thinking and emerging trends and reflection) and a production piece (a minimum length three minute narrated video showing the completed practical piece).

The focus of this project will be the book ‘The Very Hungry Caterpillar’ (Carle, 1969) and I will use AR technology to recreate this book in an interactive way that uses the technology to promote engagement with the end user.

This is a book that I have at home that I bought previously, and it was used as reference throughout this project for the image and animation content.

Idea Generation and Research

Before starting this project, I decided to see if AR was commonly used in children’s books already and found that a common theme for AR in children’s books was the use of 3D and real-life motion.

A primary platform for existing content was through the use of an app with the physical book, which makes the content easily accessible with the use of a phone. For this project, I decided to experiment with the use of 2D animation and AR because this was not a common theme with existing AR when in conjunction with children’s books. By testing this, it will give insight into if 2D animation is less commonly seen because of the effectiveness in terms of engagement or if it could be less commonly seen because AR is often linked with 3D because the primary purpose of AR is to connect with reality (Holt, 2023).

A recent study observed that when presented with an AR storybook and a print storybook, the AR had a positive influence on the reading comprehension of the students (Danaei et al., 2020). Although the AR had a positive influence on the reading comprehension of the children in this study, it does not consider if the same effect would apply for different ages and subject of the book, or for both 2D and 3D motion. It also does not consider if perhaps sound would impact the influence of AR to reading comprehension.

Due to the lack of research into the subject of my project, I will have to attempt to monitor the predicted results throughout with little evidence to show if the project could be considered effective. If this project were to expand beyond this module, user testing and further research would need to be applied to fully evaluate the effectivity of using 2D animation in AR to increase both reading comprehension and engagement for specific age categories in children.

Project Management

To start this project, I attempted to following the plan shown on my Gantt chart first created for the proposal for this project.

For the storyboard I created a simple, hand drawn selection of the scenes for this project. Originally displayed in my proposal, it shows how the movement of the assets will interact when using AR. I then moved onto the creation of visual assets.

Using the book as reference, I went through and drew the assets that I needed in Procreate in a similar style with a rough paint brush stroke. Although this worked well for the production of images, I would be inclined to export the images as I went in the future because when I went to export the images, I was unable to export them all at once and it became quite time consuming to have to go into each file individually to export it.

I then moved onto creating the animations. I isolated the movement of the caterpillar character and added a guide so I was able to use the movement and alter the path for several scenes. I then went through scene by scene to create the animations. Once the animations had been created, I exported them as PNG sequences with alpha and zipped the file ready to import into Adobe Aero.

I then added the animation to adobe aero, using the page background as the image anchor. The behavior was then edited to ensure that the animation would start and the AR was tested using the mobile app.

In terms of project management, I struggle to keep to the timescales listed in my Gantt chart, which meant I was spending long days attempting to keep on track. This caused some of my work to be of lesser quality and take more time. In future, I should plan to have regular checks on my progress and create timescales for smaller tasks, so that the milestones seem less overwhelming.

One way I could have done this is by adding deadlines to my task management board that I created for the proposal. This could have assisted in making the milestone tasks easier to follow and keep track of.

Software Proficiency

Imagery:

Procreate is an app available for a one-off fee that allows artists to create work digitally, with a range of pre-set brushes available and the option to create your own (Procreate, 2024). There are features such as reference photos, importing images and smudging to give the user a real life experience in a virtual canvas.

Using a variety of colours, I attempted to keep the imagery I created for this project similar to the original, which included the use of different shades of colour with a rough brush. Once I’d created the images, I then exported them to OneDrive with a transparent background ready for the animation.

Animation:

Adobe Animate is a program we were initially introduced to in a previous module (Adobe, 2024). Since, then I have explored how Adobe Animate can be used for different types of animations. Adobe animate works similarly to illustrator in terms of creating assets within animate itself, but has the capabilities to allow the user to easily create 2D animation using the different tools. To create the movement of the caterpillar when travelling from one place to another, I imported the layers of the caterpillar and converted it to a symbol. I then went into the symbol layer and created the general movement of the caterpillar frame by frame. I then exited back to scene and attached the symbol to a guide path, which showed Animate point A and point B for the caterpillar.

AR:

In this module we were originally taught to use Unity and Zapworks for AR. Before starting this project I considered the strengths and weaknesses of Zapworks and if there were alternatives to this that might be more beneficial.

This video shows an overview of some of the functions of Zapworks. With Zapworks you are able to do both the AR and animation/movement in one program and it has extensive tools to create more advanced projects. The main limitation to this software is the pricing; Zapworks can cost anything from £9.99 to £200 per month depending on needs (Zapworks, 2024), which is extremely expensive especially when it is unlikely to be the only program that the companies using this will be paying for.

Another alternative is Adobe Aero.

Adobe Aero has limited functions due to its recent release, but using the tools they have available, users can create AR creations without the use of coding. The program seems extremely straight forward with a predictable UI and has the benefit of being free.

When I considered the needs of this project, I found that both programs would easily fulfil the needs of the AR proportion and ultimately decided on Adobe Aero. By using this, if the project was made public, it would keep the costs down and ultimately make the product more accessible to the public in terms of cost.

Following this decision, I began to explore Adobe Aero and how it works to produce AR with an image anchor.

Once I understood the basics of Adobe Aero, I attempted to import one of the animations I’d created into it. When it didn’t work, I realised that the animation needed exporting from Animate in PNG sequence format and importing in a zipped folder to Aero. I did this and then it still wasn’t working when I previewed the AR. I then realised that I needed to tell Aero to run the sequence when the AR starts. Once I did this, I then tested the animations with the anchor and they worked as expected.

Ethics and Values

Following the proposal and the ethics considerations I considered, I have thought about what other ethical considerations could apply to this project and realised I had not considered the risk of data collection. The AR element of this project involves young children allowing the device access to their camera, which could impose a serious safeguarding and ethical issue. Should this project go live to the public, the app would need parental supervision during use and it would need to be protected against any threats.

In terms of the ethical consideration of copyright, I made it clear on the front page design that the book is an adaptation of Eric Carle’s original concept and I sought advice from staff at the university if it was something I would be allowed to do as a proof of concept to test viability and that was confirmed.

Forward-Thinking and Emerging Trends

In previous years we have seen the rapid advancement of AR both in the retail sector and entertainment sector, which have given the average person an insight into the use of AR. AR has even assisted in the medical sector allowing surgeons the ability to plan surgeries and treatments, and even in the role of educating patients in their own conditions (Eckert, Volmerg and Friedrich, 2019). AR technology is beginning to become more accessible in terms of price point and stock to the general public and with the increase of games using AR (Such as Pokemon Go (Pokemon, 2024)), it is likely that the AR technology will become further advanced and more commonly used as time progresses.

Alternatively, there could be some uncertainty surrounding the use of AR by the general public.

One show that shows the potential dark side of augmented and virtual reality is the show Black Mirror (Black Mirror, 2011). This show looks at emerging technologies and creates engaging short stories on what could go wrong when the technology becomes too advanced or is not used for purpose. One of the episode named ‘Playtest’ follows a young man who agrees to test a new immersive augmented reality game that is intended as a therapeutic method to expel fears. Towards the end of the episode, it appears as though he has left the game when he returns home and his mother calls him. Unfortunately, unbeknownst to the viewer and the main character, he is still actually in the game and the call from his phone in reality caused a short in the VR connection and ultimately kills him.

Entertainment that uses new technology as the forefront in a negative light could cause anxiety for the general public when trying to convince them to embrace the upcoming and emerging technology. This could be evidenced by a recent study that showed that although the public have a 74% trust rating in technology as a whole, only 50% trusted the technology for AI (Kitchen, 2024). Although AI is a different technology to AR, they are both emerging technologies which could make the study apply towards AR to an extent. Another point to note is that with the sudden increase in AI potential we could be leading towards technology that combines both AI and AR, which could mean a drop in the general public’s trust in emerging technologies as a whole.

Reflection

Reflecting on this project, initially I was surprised by how many ethical considerations there was for the application of emerging technologies, which made me consider if there was considerations I’d missed in previous work. In future I will ensure I thoroughly research potential ethical considerations for each project and ensure I am thorough when looking into further considerations to ensure that I don’t miss anything.

I was also pleasantly surprised at how easy the interface was for the AR technology. Previously I have been reluctant to experiment with AR as I assumed it would be extremely complicated, but the software provided by Adobe was simple and straight forward.

In terms of the success of the project, overall I feel the project has shown that with the use of emerging technologies we are able to make children’s books more interactive for the end-user even with 2D animation, rather than 3D as expected. The book allows the end-user to watch the animation of the page and also practice their reading comprehension and it has made me question why AR and 2D animation have not been used as a tool for children’s books more commonly.

I did realise following this project that 2 of the pages had the same background for the image anchor, which would encounter problems practically if the AR element used an app. This would need to be resolved prior to the project going live.

The following are the QR code links to each page (Page 1 and 16 are images as they are the front and back pages):

Page 1

Page 2 and 3:

Page 4 and 5:

Page 6 and 7:

Page 8 and 9:

Page 10 and 11:

Page 12 and 13:

Page 14 and 15:

Page 16:

Reference list

Adobe (2023). Creating Augmented Reality | Getting to Know Ar in Adobe Aero | Creative Cloud. [online] www.youtube.com. Available at: https://www.youtube.com/watch?v=jAfR_JY3P5g.

Adobe (2024). Buy Adobe Animate | Flash and 2D animation software. [online] www.adobe.com. Available at: https://www.adobe.com/uk/products/animate.html.

Black Mirror, (2011). Charlie Brooker.

Carle, E. (1969). The Very Hungry Caterpillar. London: Puffin.

Danaei, D., Jamali, H.R., Mansourian, Y. and Rastegarpour, H. (2020). Comparing reading comprehension between children reading augmented reality and print storybooks. Computers & Education, 153, p.103900. doi:https://doi.org/10.1016/j.compedu.2020.103900.

DigitalDefynd, T. (2024). 100 Inspirational Quotes About AR & VR Technology [2024]. [online] DigitalDefynd. Available at: https://digitaldefynd.com/IQ/inspirational-quotes-about-ar-vr-technology/#:~:text=2. [Accessed 21 May 2024].

Eckert, M., Volmerg, J.S. and Friedrich, C.M. (2019). Augmented Reality in Medicine: Systematic and Bibliographic Review. JMIR mHealth and uHealth, 7(4), p.e10967. doi:https://doi.org/10.2196/10967.

Holt, N. (2023). Augmented Reality and Virtual Reality in 3D Modeling. [online] RebusFarm. Available at: https://rebusfarm.net/blog/augmented-reality-and-virtual-reality-in-3d-modeling.

Kitchen, K. (2024). Innovation in Peril: A New Survey Shows Global Trust in Technology is Declining. [online] www.linkedin.com. Available at: https://www.linkedin.com/pulse/innovation-peril-new-survey-shows-global-trust-klon-kitchen-t0u9e#:~:text=For%20example%2C%20the%20technology%20sector [Accessed 22 May 2024].

Pokemon (2024). Pokémon GO. [online] Pokémon GO. Available at: https://pokemongolive.com/?hl=en.

Procreate (2024). Procreate® – Sketch, Paint, Create. [online] Procreate. Available at: https://procreate.com/.

Zappar (2016). ZapWorks: The AR, VR & MR content creation tool. [online] YouTube. Available at: https://www.youtube.com/watch?v=zgkGDSQTy1g [Accessed 21 May 2024].

Zapworks (2024). Zapworks Pricing: Choose The Right Plan For You. [online] zap.works. Available at: https://zap.works/pricing/.