Enhance your table tennis skills

By Philip Buthmann (phbut19) and Asger Nybo (asnyb15)

Complete code: https://bitbucket.org/philbudi/191205topspin/src/master

Video of application

Outline

1. Introduction
2. Method & Materials
3. Results
4. Discussion

Introduction

This report covers the making of an IOS App, using the SDU sportsensor, as it had a lot of potential. Through brainstorm, it was decided to make an app that resolved around table tennis, as Philip is a table tennis player. The idea was to make an app that helps improve ones table tennis skills, by tracking the movement of the players hand with an external sensor connected via bluetooth.

Research showed that the market for a table tennis app using external sensors were very limiting, thus making it worthwhile to persue filling a gap in the market and introduce new ways of improving ones table tennis skills.

Problem:
Can you make an application that tracks your table tennis training sessions and help you become a better table tennis player?

How:
It is intented to strap the SDU sportsensor to a wristband attached to the player, which aquire accelerometer data about the players hand movement, which will be processed by the application and showed to the user in a userfriendly way.

Methods and Materials

This section describes what was done and what materials were used to accomplish the making of the desired application. The first thing there had to be done was to make sure what the actual desired application was, which was done through a brainstorm.

Brainstorm

A brainstorm was conducted by discussing all the possibilities of the SDU Sportssensor. This included ideas such as making an apps for strengthtraining by tracking the athletes performance through accelerometer data, making an app for monitoring people posture by attaching the sensor to the back and process the data. All ideas was written down, and a filtering was done in the end.

Conceptual design

Next a conceptual design was made by writing a list of what was wanted the design of the chosen application to include.

Use-Case diagram

Furthermore a use-case diagram was constructed to specify the use case and making it clear how the application was intented to be used.

Prototype

A prototype was made mainly in PowerPoint to visualize the design that was beforehand only conceptual. PowerPoint gave the advantages of making the prototype somewhat funtional, by making working buttons to switch screen and simulate data acquisition by changing numbers ect. from slide to slide.

Evaluation

An evaluation was done based on the feedback recieved from a presentation of the prototype to the class.

Requirements

Based on the evaluation and the previuos work, a table was made of the requirements for both the model and the views. The requirements were split up into two categories: must and optional. “Must” meaning that the given requirement is considered a must and “optional” meaning that the given requirement i nice to have but not a need to have.

Object/UML-class Diagram

Based on the requirements an object/UML-class diagram was done to make it clear what the structure of the application should be, to fulfill the requirements.

Implementation

Now having layed out the foundation for the desired application, the implementaion began.

Final Evaluation

After implementing as much as possible in the timeframe of the project, a final evaluation was made. This implied giving the sensor and the application to some persons who had no knowledge about the application other than the overall usecase.

Results

This section covers the results of the methods and materials used.

Brainstorm

The result of the brainstorm was the idea of making an application for table tennis.

Conceptual Design

List of things wanted for the design
1. Three different views (Live data, Statistics, Settings)
2. Live data should show total hits, max accelerations, and last hit acceleration
3. Statistics should show the statistics of the last training session, preferbly in a graph
5. Settings should show include settings to adjust the application to the user

Use-Case diagram

The use-case diagram that was made is shown below

Get live feedback:
The user should attach the sensor and connect bluetooth in order to get live feedback
View Training Statistics:
Here the user can see the statistics about their training session
Settings:
The user is able to change settings for the application

Prototype

The result of the prototype made in PowerPoint is shown below.

Seen from left to right the first screen is how the start screen is intented to be presented, screen number 2 shows when the play has pressed start session and hit the ball once. Screen number 3 shows the screen after 5 hits where it has calculated the last stroke speed, average stroke speed and the maximum stroke speed. Screen number 4 shows the statistic screen with a graph over the different stroke speeds, with some additional data.

Evaluation

The feedback from the presentation included:
Reflection on why we didnt use apple watch instead of an new device.
Confirming the likeablility of the general idea and functions presented.

The reason why an apple watch isn’t used is because it is expensive, if you only want to use it for this application, which an sensor made for only that might not be.

The good feedback on the prototype strengthened the confidence in the concept and design, and was therefore kept.

Requirements

Beneath is the tables showing the requirements made for the application

High level Requirements (Model)

No.	Description	Must	Optional
M1	Counter of the Racket hitting the ball, e.g. in a period short term	x
M2	max speed, top acceleration decline, e.g.  in multi ball training	x
M3	Store the data → measure your improvements, long term	x
M4	Set a minimum & max. (sweet spot racket speed) → instant feedback by turning the screen red or green		x
M5	Differ between top spin, counter, slice		x
M6	Visualize the whole Movement		x
M7	Professional “advice” by comparing the data to your own		x

Requirments for the Display Views (View)

No	Description	Must	Optional
V1	Start Screen (Menu; 3 Modes)	x
V2	Settings	x
V3	View Mode 1: Counter with Speed, Real Time Data	x
V4	View Mode 2: Progress things with graphs (apple health app inspiration)	x
V5	View Mode 3: Visualization of movement		x
V6	Making different Users/ Profiles		x

These will be commented on in the discussion wether they were fulfilled or not.

Object/UML-class Diagram

The final UML- class diagram contains the tab-bar navigation controller presenting the four view controller TrainingViewController, StatisticsViewController, ProModeViewController and InstructionsViewController. The two last mentioned ViewController are not containing any changeable functions or features at the moment. The Instructions had been designed with the storyboard and contain a picture.

Initially it has been planned to establish the bluetooth connection as a separate model which had to be reconsidered since the bluetooth service subscription is handled in iOS always over an active view controller. All bluetooth subscriptions and handlings are contained in the TrainingViewController and taking advantage of the built in CBPeripheral functions and CentralManager functions. The TrainingViewController uses the model SensorCalculations to create arrays of the received data and the unwrapping of the receiving Bytes from the SDU bluetooth sensor. Statistics are displayed in the StatisticsViewController in a graph which is created out of the data receiving from the SensorCalculations.

Implementation

The application is build upon mainly three files:
TrainingViewControler, SensorCalculations and StatisticsViewController.
The main functions of these files will be explained below

TrainingViewController/Bluetooth

This files handles what is shown in the TrainingView, and the bluetooth connection. The bluetooth connection was done primarily with help from https://www.raywenderlich.com/231-core-bluetooth-tutorial-for-ios-heart-rate-monitor

func centralManager(_ central: CBCentralManager, didDiscover peripheral: CBPeripheral,
                        advertisementData: [String: Any], rssi RSSI: NSNumber) {
        
        
        let deviceName = NSString("BC05FC978B7510AB")
        
        let nameOfDeviceFound = (advertisementData as NSDictionary).object(forKey: CBAdvertisementDataLocalNameKey) as? NSString
        
        
        print("\(String(describing: nameOfDeviceFound))")
        
        if nameOfDeviceFound == deviceName  {
            // Update Status Label
            self.connectedLabel.text = "SDU Sensor connected"
            
            // Stop scanning
            self.centralManager.stopScan()
            // Set as the peripheral to use and establish connection
            self.accelerationPeripheral = peripheral
            accelerationPeripheral.delegate = self
            self.accelerationPeripheral.delegate = self as? CBPeripheralDelegate
            self.centralManager.connect(peripheral, options: nil)
        }
        else {
            
            self.connectedLabel.text = "NOT Found"
        }
    }

CBCentralManager is an object that scans for, discovers, connects to and manages peripherals. Peripherals being any external device that provides input and output for a computer (in our case Peripheral is the sportsensor and the computer is an Iphone).
The above code calls a funcion “didDiscoverPeripheral” when a peripheral is discovered. It then checks if the name of the peripheral is equal to the sportssensors name, and if so it connects to the device.

 func centralManager(_ central: CBCentralManager, didConnect peripheral: CBPeripheral) {
        print("connected")
        print(peripheral)
        peripheral.discoverServices([kBandCizerServiceUUID])   
    }

When a device is connected the function “didConnect” gets called. In here another function gets called, which discovers the services in the peripheral.

    func peripheral(peripheral: CBPeripheral!, didDiscoverServices error: NSError!) {
        
        for service in peripheral.services! {
            let thisService = service as CBService
            if service.uuid == kXAccCharUUID {
                // Discover characteristics of bandCizer thing
                peripheral.discoverCharacteristics(nil, for: thisService)
            }
            print(thisService.uuid)
        }
    }

If it discovers some services, it checks if they are equal to the XAccCharUUID, which is a UUID(universally unique identifier) for the accelerations. This UUID is gathered from the datasheet of the sportssensor.
It then discovers the characteristics for the specified service.

 func peripheral(_ peripheral: CBPeripheral, didDiscoverCharacteristicsFor service: CBService,
                    error: Error?) {
        guard let characteristics = service.characteristics else { return }
        
        for characteristic in characteristics {
            print(characteristic)
            peripheral.readValue(for: characteristic)
            peripheral.setNotifyValue(true, for: characteristic)
            
        }

When the characteristics is discovered it gets telled that we would want to read from it and that it should notify when a new value is recieved.

    func peripheral(_ peripheral: CBPeripheral, didUpdateValueFor characteristic: CBCharacteristic,
                    error: Error?) {
        
        switch characteristic.uuid {
        case kXAccCharUUID:
            
            let accX = SDUSensorCalc.accelerations(from: characteristic)
            onAccelerationsReceived(accX)
       
        default:
            _=0 //print("")//Unhandled Characteristic UUID: \(characteristic.uuid)")
        }
    }

When a characteristics value is updated the above function gets called. This function checks with a switch case if its the kXAccCharUUID that has been updated, and in that case it calls two functions called SDUSensorCalc.accelerations() and onAccelerationsRecieved().
SDUSensorCalc.accelerations() is handled in the SensorCalculationsModel.
onAccelerationsRecieved() is seen below

 func onAccelerationsReceived(_ accelerationArray: Array<Double>) {
            
            if speedZ.isHidden == false {
                speedZ.text = String(format: "%.3f",accelerationArray[2])
                speedY.text = String(format: "%.3f",accelerationArray[1])
            }
            
            if TrainingViewController.trainingActive == true {
                           
                swingSpeedArrayText = SDUSensorCalc.swingSpeed(accelerationZ: accelerationArray[2])
                
                lastHitAcceleration.text = String(format:"%.3f",swingSpeedArrayText.last ?? 0.0)
                
                maxAccValue = swingSpeedArrayText.max() ?? 0.0
                
                maxAcc.text = String(format: "%.3f", maxAccValue)
          
                totalHits.text = String(SDUSensorCalc.SavedSpeeds.count)
                
            }
}
        

Whenever a new acceleration value is recieved it processes and calculates interresting variables for the user, which gets set to textlabels here to show the in the view.

SensorCalculations/Model

The model sensor calculation calculates all acceleration values and stores them in arrays if a certain treshhold is exceeded. The length of the resulting array represents the number of hits the user has made.

    func accelerations(from characteristic: CBCharacteristic) -> Array<Double> {
        guard let characteristicData = characteristic.value else { return [0.0] }
        
        let g_pr_lsb = 0.000244
        var g_xdata = 0.0
        var g_ydata = 0.0
        var g_zdata = 0.0
        
        let byteArray = [UInt8](characteristicData)

        let accXBit16 = (Int16(byteArray[3]) << 8) + Int16(byteArray[2])
        let accYBit16 = (Int16(byteArray[5]) << 8) + Int16(byteArray[4])
        let accZBit16 = (Int16(byteArray[7]) << 8) + Int16(byteArray[6])
        
        g_xdata = g_pr_lsb * (Double(accXBit16))/4
        g_ydata = g_pr_lsb * (Double(accYBit16))/4
        g_zdata = g_pr_lsb * (Double(accZBit16))/4
        
        let g_forceArray = [g_xdata,g_ydata,g_zdata]
        return g_forceArray
    }

The function “accelerations” gets the bluetooth characteristics data via a call from TrainingViewController and processes the received bytes, converts them to an Int16. The g-force value can be up to 2Gs with this sensor. Every time the function gets called, the converted values in every direction gets wrapped in an array which is returned.

   
    func swingSpeed(accelerationZ: Double) -> [Double] {
        
        let currentTimeSwing = CFAbsoluteTimeGetCurrent()
        
        if accelerationZ > 1 {
            SwingSpeedArray[1] = accelerationZ
            
            print("1:\(SwingSpeedArray)")
            
            if SwingSpeedArray[1] < SwingSpeedArray[0] && (currentTimeSwing-SavedTimeSwing) > 0.6 {
                
                SavedTimeSwing = CFAbsoluteTimeGetCurrent()
                
                SavedSpeeds.append(SwingSpeedArray[0])
                
                print("2:\(SavedSpeeds)")
                SwingSpeedArray[0] = 0.0
                SwingSpeedArray[1] = 0.0
            }
            
            if (currentTimeSwing-SavedTimeSwing) < 0.4 {
                
                SwingSpeedArray[0] = 0.0
                SwingSpeedArray[1] = 0.0
                
            }
            
            SwingSpeedArray[0] = SwingSpeedArray[1]
        }
        
        //let lastNumber = SavedSpeeds.last ?? 0.0
        
        SensorCalculations.lastSpeedsArray = SavedSpeeds
        
        return SavedSpeeds
    }

The function swingSpeed saves the acceleration values over a certain treshhold and a certain time period. Only from the moment when the last value was higher than the current value the last value is stored. This represents the methodology just to get the highest values of them.

StatisticsView

The StatisticsViewController displays the current statistics like the average and the total hits. It is possible to view the statistics during an active training session. Everytime the tab Bar “Statistics” gets called, the view refreshes.

   override func viewDidAppear(_ animated: Bool) {
        
        statisticsArray = SensorCalculations.lastSpeedsArray
        makeGraph()
        print(statisticsArray ?? 0)
        
        currentAvg = 0.0
        
        for i in statisticsArray {
            
            currentAvg += i
        
        }
        
        arrayLength = Double(statisticsArray.count)
        currentAvg = (currentAvg/arrayLength)
        currentMax = statisticsArray.max() ?? 0.0
        
        averageHitSpeed.text = String(format: "%.3f",currentAvg)
        maxHitSpeed.text = String(format: "%.3f",currentMax)
        totalHits.text = String(format: "%.0f",arrayLength)
        
 
         
    }

The viewDidApper function gets called every time the view appears and the graph is redrawn. The array with the last speeds (accelerations) from Sensor calculations gets called and is used to calculate the current averages and the maximum acceleration of the displayed values in the graph. Additionally the textboxes are set and formatted.

 
    func makeGraph() {
        
        var lineChartEntry = [ChartDataEntry]()
 
        for i in 0..<statisticsArray.count {
         
            let value = ChartDataEntry(x: Double(i), y: statisticsArray[i])
            
            lineChartEntry.append(value)
            
        }
 
        let line1 = LineChartDataSet(entries: lineChartEntry, label: "Hit accelerations")
        
        line1.colors = [NSUIColor.blue]
        trainingChart.xAxis.labelTextColor = UIColor.white
        
        line1.circleColors = [UIColor.green]
        
        line1.drawCircleHoleEnabled = true
        line1.circleHoleColor = NSUIColor.blue
        
        trainingChart.xAxis.labelPosition = .bottom
        trainingChart.xAxis.drawGridLinesEnabled = false
        
        trainingChart.leftAxis.labelTextColor = UIColor.white
        trainingChart.leftAxis.axisMinimum = 0.0
        trainingChart.leftAxis.axisMaximum = 2.2
        trainingChart.leftAxis.drawAxisLineEnabled = false
        trainingChart.leftAxis.drawGridLinesEnabled = false
        
        trainingChart.drawBordersEnabled = true
        trainingChart.borderColor = UIColor.white
        
        trainingChart.legend.textColor = UIColor.white
        
        trainingChart.chartDescription?.textColor = UIColor.white
        
        trainingChart.xAxis.granularity = 1.0
        
        let data = LineChartData()
        data.addDataSet(line1)
        
        trainingChart.data = data
    
        print(statisticsArray.count)
        
    }

The function “makeGraph” is an add-on of the Cocoa Pods and is implemented in this project.
https://cocoapods.org/pods/Graph

The visual appearance such as all labels, colors, ranges of the axes and granularities are set here.

Complete code

https://bitbucket.org/philbudi/191205topspin/src/master/

Final Evaluation

The result of the evaluation was feedback from the subjects. This included:
– One didn’t notice the tab bar at first.
– One read the instructions but didn’t seem to use it.
– A bit confusion about when a training session was started.
– Otherwise good feedback

Discussion

The problem of the project was:
Can you make an application that tracks your table tennis training sessions and help you become a better table tennis player?
The final application was able to track stats like the acceleration of each hit, the maximum acceleration and the amount of hits.
What didnt work was the saving of data through each session and the sensor being preinstalled to only measure up to +-2g in acceleration. This meant that the acceleration maxed out pretty fast, making the data very uninterresting.

Fulfillment of the requirements

High level Requirements (Model)

No.	Description	Must?	Implemented
M1	Counter of the Racket hitting the ball, e.g. in a period short term	Yes	Yes
M2	max speed, top acceleration decline, e.g.  in multi ball training	Yes	Yes
M3	Store the data → measure your improvements, long term	Yes	No
M4	Set a minimum & max. (sweet spot racket speed) → instant feedback by turning the screen red or green	No	No
M5	Differ between top spin, counter, slice	No	No
M6	Visualize the whole Movement	No	No
M7	Professional “advice” by comparing the data to your own	No	No

M3: Obscure problems with CoreData occured
M4-7: Not enough time to implement

Requirments for the Display Views (View)

No	Description	Must?	Implemented
V1	Start Screen (Menu; 3 Modes)	Yes	No
V2	Settings	Yes	No
V3	View Mode 1: Counter with Speed, Real Time Data	Yes	Yes
V4	View Mode 2: Progress things with graphs (apple health app inspiration)	Yes	Yes
V5	View Mode 3: Visualization of movement	No	No
V6	Making different Users/ Profiles	No	No

V1-2: Was found to be unnecessary
V5-6: Not enough time to implemented

Conclusion/Further Work

Despite that most of the wanted results had been achieved further implementations would be meaningful to include. In the beginning it was planned to save the training sessions for later comparisons. Due to problems with the database and CoreData, the feature could net be implemented in time. Another feature would be a color exstension for the instant feedback, e.g. in landscape mode. Many inspirations for the future can be deducted from the App Zepp (https://www.zepp.com/en-us/tennis/highlight-reel/). Taking into account the user feedback, more attention should be given to colors and color differentiation so that certain elements could be seen better. For example the start of the training session just changes the color of the button. It might be useful to start a new ViewController in a new color after pressing the start training button. Another feature could be sliding between the screen with screen edge pan gesture recognizer.

The next development step will include some training sessions with table tennis players if the range of the accelerometer can be increased to +/-16g. 

Despite a lot challenges in the beginning with the understanding of how to connect and receive data from the bluetooth sensor the project was processed in time due to continous coding sprints and team meetings two times per week. The team members learned to implement connection to external devices via bluetooth, the whole UI of an applecation, graphs and much more.