By Niclas Kildahl Laugesen (nilau12) and Nicklas Schmidt Vejlgaard (nicha13).

Introduction

By discusses modern apps and how they are used we found that children often uses phones or iPads while sitting down. We discussed how we could take advantage of the sensors in the device to create something that demanded more physical engagement. This gave us a vision to create a simple app that forces the user to engage more physically.

The goal is to create a simple application that by tilting the device draws a line on a blank page. The line changes colors depending on which direction the device is heading. Our target users are primarily children. The act of drawing should be somewhat challenging but only just enough to make it fun. The core function of the app is the drawing this means that the menus should not too much focus. A simplistic and intuitive menu design is therefore crucial. We will measure our success based on the amount of physical activity needed to draw a figure compared to standard drawing apps.

This concept dives down in a burning topic at the moment, which is “how much time should kids be allowed to spent sitting with a tablet or phone”. This app purposes a middle ground where physical movement is required for a fun and creative experience.  

The app will use the devices onboard sensors primarily the compass and gyroscope. The compass is magnetometer from where you can access the heading of the phone compared to magnetic poles of the planet. The gyroscope is like a carpenter’s level it just measures on all three axes at once and can determine the devices angulation in relation to the gravitational force. These sensory inputs are directly accessible through the code.

The primary question for us to ask is, how can you create a drawing application that makes it challenging and fun to draw while facilitating movement? The second thing we need ask ourselves is how can we make the interaction of this application so intuitive that it will be easily accessible for children to use? These questions are the base for all our decisions. If we make a design choice we will ask if it answers question number 2 and if we are to make a decision towards the drawing experience we will ask question 1.

The answer for question one, that we have been working from so far, is to create something similar to having a ball dipped in paint, on a stiff board and tilting the board, to draw lines with the paint. Here the ball becomes the pen which we use for drawing.

 

Methods and materials

The process started with a short unstructured brainstorm on the state of mobile application and use of mobile devices. The lead us to an topic the interested us both. On the basis of this topic we did a brainstorm to establish what goal the app should achieve. From this we defined a key value, it should be a simple yet more physically engaging mobile app. This could be described as an iterative process of brainstorming that served to zoom in on a concept. We started wide by finding a problem, then a key value and finally zooming in on a solution that served our key value.  

Our solution was defined as a concept and illustrated (figure x) .

Figure 1

 

This allowed us to make some early reflections towards design, interaction and setup.

These reflections were then used to design a low-fi prototype. For our low-fi prototype we focused on trying out the challenge and enjoyability of drawing by tilting the device. For this we used a flat, rigid object to place a piece of paper on while one of us held a pen tip side down on the paper. We would then simulate the movement of the ball by moving the pen in the direction a ball would roll.

This gave us a confirmation that the concept of drawing this way was something to pursue further so we moved on to a digital prototype.

In the digital prototype the ability to draw was the primary focus yet again. Therefore the menus and other interactions was kept at a minimum and no real reflection went into the design of these.

We conducted a think aloud test with some different people both within, but also outside, the primary demographic. The users were to try out the different functions in the app while talking about their reasoning behind their actions. From this test we got a lot of useful feedback.

  • The user would like to be able to go back and continue drawing after pressing “Done Drawing”.
  • The users would like to be able to move the pen without drawing.
  • The user thought it was a fun way to draw and were very emerged in the game while testing.
  • The users tried to move the screen instead of moving themselves.

Based on this and our initial value we came up with a set of requirements.

  • Intuitive interface design.
  • Easy overview of available menu actions
    • Ie. no drop down menus or other hidden options.  
  • Different levels of difficulty
  • Possibility of saving and/or sharing drawings.

Results

Prototyping

Idea

By relying on user feedback and expert knowledge we iterated our way to a final concept. A simple welcome menu, which navigates to some choices for difficulty, a draw view and an options view. The users will have the ability to choose their own difficulty to make sure that the drawing stays challenging with becoming too challenging. The draw view and welcome view is kept as simple as possible, there is however the possibility of adding more options to both views. The options view has five different icons representing the five different options the user has after creating a drawing. We chose to go with icons that this gives a faster interaction time then words. The different functions are save, share, delete, continue and home.

Navigating the application

The first navigational prototype contained three views; an introduction view which leads to a drawing view, when the user is done drawing she can go to the end view.

 

Introduction view

In the introduction view the title of the app is displayed and below that is two options allowing you to choose how to draw. The two options were real physics or fake physics. Both choices led to the same draw view. However the ball would behave differently depending which type of physics was chosen.

 

Drawing view

In this view there is only ball and drawing that it creates visible along with a small button in the upper hand corner with the text “Done Drawing”. This button leads to the end view. The ball i controlled by taking sensory input from the device’s gyroscope and the color is changeable by pointing the device in different directions.

 

End view

In this view, the user is presented with a variety of options. The most of the screen is filled with an image of the drawing, the user has created. along the bottom and up the right hand side. Is a set of buttons including, save, share, delete, continue drawing and main menu.

Each buttons description logically informs the user of their possible options.

In later iterations we decided to make four views instead of three, because we wanted to limit the number of buttons per view, and because we implemented the functionality to continue your last drawing if you quit the application another button needed to be added to the introduction view. So instead of having three buttons on the introduction page we decided on keeping two, one for starting a new drawing and one for continuing the last drawing. Then when you press the start drawing button you will be presented with the new view where you choose how you want the ball to control.

Navigating the application

As mentioned there were three views in the first prototype. You switch between these three views by pressing buttons. Pressing a button will trigger an action segue which will modally present the next view. This was implemented by ctrl-dragging in the storyboard.


Implementation

The images beneath (Figure 2 and Figure 3) shows the attributes and the functions implemented in the various view controller, furthermore, the second image shows the functions and variables of the implemented classes that are not view controllers.

Figure 2 – Depiction of the flow of the views and their contents
Figure 3 – Depiction of the contents relationships of the non view controller classes

Navigation

Navigation is changed from the action segues that were implemented in the navigation prototype, since this worked fine for the purpose. However, we need to pass data from one view to the other in order to do that, we need to use the UIViewController function

override func prepare(for segue: UIStoryboardSegue, sender: Any?)

This function is called right before the segue to the next view happens(https://developer.apple.com/documentation/uikit/uiviewcontroller/1621490-prepareforsegue), this is a good time to set data to the next view. In order to push the data, the view controller must have variables to store the data. You can then access and assign data to these variables from one view to another. The way you access the destination view is by accessing the segue parameter of the aforementioned functions. This UIStoryBoardSegue contains a pointer to the destination view controller. The code looks like follows:

if let viewController = segue.destination as? DrawViewController{
    if let drawn = drawing {
        viewController.drawing = drawn
    }
}

 

What happens here is that we try and set the variable viewController to the destination of the segue while trying to typecast the destination as a specific object in this case a DrawViewController object. This enables us to check what view the destination is, because if typecast fails, it will not execute the code within the if statement. This is for example useful for the end view because from there we are able to go to both the drawing view and the introduction view, and we need to only pass the drawing to the draw view.

In the segue from the introduction view to the main screen view (sorry for the bad naming convention, this is the view where the user decides how the ball controls) we push the drawingStateStore. The drawing state store is the object responsible for storing the last drawing state so that the user can continue drawing it the next time she opens the app. More on this in the accessing the last drawing section. Actually we need to push this DrawingStateStore everytime we segue to a new view, otherwise the object will be erased from memory and will therefore become inaccessible. So even though we will not mention that this is pushed for the other segue it will be pushed.

In the segue from the introduction to the drawing view, we push: how the ball controls (marblePhys, also a bad naming convention), the last drawing and the pen (see the image that shows the classes for more information).

In the segue from the main screen view to the drawing view we push how the ball controls.

In the segue from the drawing view to the end view we push the drawing, the pen and how the ball controls. The reason for passing these this is that the draw view will be erased from memory when we segue to the end view, so we cannot simply return from the end view and continue, we have to push these objects back and forth. This might not be the best design but for now it works. This also means that in the segue from the end view to the drawing view we push the same data.

In the segue from the end view to the introduction view we push no data, except for the drawingStateStore of course.

 

Moving the pen

In order to move the pen by tilting the device, we need to import CoreMotion, from which we get the class coreMotionManager. In the viewDidLoad function we set initialize an object of that class and sets the update interval and decides what happens at each update. In each update we get the accelerometer data, we use this to update the position of the pen, and then draw a line on the tempImageView.image from the last point to the current one.

We update the position of the pen using the following code:

xPos += xVel * deltaTime + 0.5 * accelerationX * deltaTime * deltaTime
xVel += 750 * accelerationX * deltaTime
yPos += yVel * deltaTime + 0.5 * accelerationY * deltaTime * deltaTime
yVel += 750 * accelerationY * deltaTime

The basis for this code is the kinematic equation for displacement over time, the accelerometer information is put in as the acceleration.

We draw a line based on the information from this website: https://www.raywenderlich.com/87899/make-simple-drawing-app-uikit-swift, but instead of using our fingers to draw we use the position of the pen.

Furthermore, we make sure the pen cannot move beyond the screen, we do this saying that the pen cannot move beyond the size of the tempImageView.

In the easier control scheme of the pen we use the accelerometer information as the velocity instead of the acceleration in the aforementioned equations, this means that the pen does not carry momentum.

 

Changing the color of the pen

We decided to change the color of the pen according to the compass heading of the device. This means that we had to figure out a way to define distinct colors using a numbers between 0 and 360. We decided to use the HSB color picker scheme, see the figure 4. This means that if we have predefined the saturation and the brightness we can set the hue using the compass heading. This gives us the distinct colors that we are looking for.

Figure 4 – Depiction of HSB color format. Source: http://www.tomjewett.com/colors/hsb.jpg

In order to access the compass information we need to import CoreLocation, and we also need to change the drawing view to inherit from CLLocationManagerDelegate. In the viewDidLoad we initialize an object of the class coreLocationManager we set its delegate to self and call its function startUpdateHeading. We also need to implement a function called locationManager like so:

func locationManager(_ manager: CLLocationManager, didUpdateHeading newHeading: CLHeading)

The code within this function will execute each time the heading is changed, so this code takes the updated heading and updates the color of the pen.

 

Accessing the last drawing

In order to access the last drawing and continue where we left off, we need to store all information necessary for this. This information is: the drawing, the pen and how the ball controls. Therefore, we have created a class that has three variables: UIImage, Pen and Bool. We also need to be able to store the information so it is accessible after we close the app, for this we used the information from chapter 16 of the book: iOS programming – The Big Nerd Ranch.

The difference is that we are not using a navigation controller so we need to push the DrawingStateStore everytime we segue to a new view.

We update the state of the drawingStateStore everytime we access accelerometer information. Furthermore, we save the state to the device when the application enters the background. This means that we need to access the appDelegate file and initialize the drawingStateStore in there and from there pass it to the root view controller. Furthermore, within the appDelegate there is a function called applicationDidEnterBackground here we call the function of the drawingStateStore that save the state to the device.

 

Screen shots of the final application

Introduction view

Main screen view

Drawing view

End view

Discussion

Throughout the process we have maintained a focus on our key value. This is evident in the final iteration of the application. We have managed to implement all of our requirements and have taken the user inputs into consideration.

The process of programming the application has been done in an iterative process where we have identified the next feature which we felt was the most important in order to maintain a functional application. For example we created the navigation first, since this was the basis of having a functional app, next we wanted to be able to pass information from view to view to view and so on. Furthermore, we were constantly evaluating the application and coming up with new ideas. We merely programmed based on sketches and by programming, we did not do any in depth planning, like making class or UML diagrams and such. This actually worked quite well, this most likely had something to with the fact that it is a fairly simple application in regards to data flow, so there is not a lot to keep track of at once.

We can conclude that we have achieved our goal of creating more physical interaction in drawing apps and we have maintained a simplistic interaction were only the most necessary is available. The users engaged with the app immersively and enjoyed the added physical aspect. However the desired effect with color change did not provide as high an amount physical activity as hoped. Though they did show a much more physical engagement with the device then with standard drawing apps. We consider our app successful however there is always room for improvement. Future development would go into creating new functionalities that would demand even more physical activity from the user.

 

Link to code: https://bitbucket.org/iosprogrammingsdu/tiltapen

Video: https://youtu.be/yZyz5W1f3AE

Leave a Reply