iPhone Programming
CS4347 Sound and Music Computing Zhou Yinsheng ([email protected]) Jan 22nd, 2011
Acknowledgement: Some materials are borrowed from the course of iPhone Application Development in Stanford University.
Total 67 Pages 1
Tools to create Apps in iPhone
Xcode
Interface Builder
Mac
Instruments Simulator/device
Total 67 Pages 2
Sound and Music Computing
We use iPhone to make music! Mobile Music Group
MOGFUN MOGCLASS MOGHEALTH
Total 67 Pages
MOGCLASS
MOGCLASS DEMO
Total 67 Pages
MOGCLASS
Do you want to create your own iPhone App or develop novel user interface for MOGCLASS?
Total 67 Pages
You need to know
Language: Objective-C and C/C++ Objective Oriented Programming Programming and testing in Xcode Digital Signal Processing basics Assembly Code (ARM) Computer Network Etc
Total 67 Pages
Today we will cover
Objective-C basics Building an Application Sensor programming (accelerometer, GPS, microphone) Audio Libraries
There are a lot of materials about iphone online. What we will cover today is more oriented for your final project.
Total 67 Pages 7
How to learn iPhone Programming
Recommended Book: None! Well use Apple documentation iOS Developer Center:
http://developer.apple.com/devcenter/ios/index.action
Download the sample code and learn.
Write the code by yourself Trial and error.
Total 67 Pages
iPhone SDK Technology Architecture
Cocoa Touch Media
Multi-Touch Events Multi-Touch Controls Accelerometer View Hierarchy Localization Core Audio OpenAL Audio Mixing Audio Recording Video Playback Collections Address Book Networking File Access SQLite OS X Kernel Mach 3.0 BSD Sockets Security Alerts Web View People Picker Image Picker Camera JPG, PNG,TIFF PDF Quartz (2D) Core Animation OpenGL ES Core Location Net Services Threading Preference URL utilities Power Mgmt Keychain Certificates File System Bonjour Total 67 Pages 9
Core Services
Core OS
Outlines
Objective-C basics Building an Application Sensor programming (accelerometer, GPS, microphone) Audio Libraries
Total 67 Pages
10
OOP Vocabulary
Class: defines the grouping of data and code, the type of an object. Instance: a specific allocation of a class. Method: a function that an object knows how to perform. Instance Variable (or ivar): a specific piece of data belonging to an object.
Total 67 Pages
11
OOP Vocabulary
Encapsulation
Keep implementing private and separate from interface
Polymorphism
Different objects, same interface
Inheritance
Hierarchical organization, share code, customize or extend behaviors
Total 67 Pages
12
Objective-C
Strict superset of C
A very simple language, but some new syntax Single inheritance, classes inherit from one and only one superclass Protocols define behavior that cross classes Dynamic runtime Loosely typed, if youd like
Total 67 Pages 13
Mix C with ObjC Or even C++ with ObjC (usually referred to as ObjC++)
Class and Instance Methods
Instance respond to instance methods
-(id)init; -(float)height; -(void)walk;
Classes respond to class methods
+(id)alloc; +(id)person; +(Person*)sharedPerson;
Total 67 Pages
14
Message Syntax
[receiver message]; [receiver message:argument]; [receiver message:arg1 andArg: arg2]
Total 67 Pages
15
Terminology
Message expression
[receiver method:argument]
Message
[receiver method:argument]
Selector
[receiver method:argument]
Method
The code selected by a message.
Total 67 Pages
16
Dot Syntax
Objective-C 2.0 introduced dot syntax Convenient shorthand for invoking accessor methods
float height = [person height]; float height = person.height; [person setHeight: newHeight]; person.height = newHeight;
Follows
the dots
[[person child] setHeight:newHeight]; //exact the same as person.child.height = newHeight;
Total 67 Pages 17
Dynamic and static typing
Dynamically-typed object
id anObject
just id Not id * (unless you really, really mean it)
Statically-typed object
Person *anObject
Objective-C provides compile-time, not runtime, type checking Objective-C always uses dynamic binding
Total 67 Pages 18
Selectors identify methods by name
A selector has type SEL
SEL action = [button action]; [button setAction: @selector(start:)];
Conceptually similar to function pointer Selectors include the name and all colons, for example:
-(void) setName:(NSString*)name age:(int)age;
would have a selector:
SEL sel = @selector(setName:age:);
Total 67 Pages
19
Working with selectors
You can determine if an object responds to a given selector
id obj; SEL sel = @selector(start:) if ([obj respondsToSelector:sel]){ [obj performSelector: sel withObject:self]; }
This sort of introspection and dynamic messaging underlies many Cocoa design patterns
-(void)setTarget:(id)target; -(void)setAction:(SEL)action;
Total 67 Pages 20
Working with Classes
You can ask an object about its class
Class myClass = [myObject class]; NSLog(@"My class is %@", [myObject className]);
Testing for general class membership (subclasses included):
if ([myObject isKindOfClass:[UIControl class]]) { // something }
Testing for specific class membership (subclasses excluded):
if ([myObject isMemberOfClass:[NSString class]]) { // something string specific }
Total 67 Pages 21
Working with Objects
Identity v.s. Equality Identitytesting equality of the pointer values
if (object1 == object2) { NSLog(@"Same exact object instance"); }
Equalitytesting object attributes
if ([object1 isEqual: object2]) { NSLog(@"Logically equivalent, but may be different object instances"); }
Total 67 Pages
22
-description
NSObject implements -description
- (NSString *)description;
Objects represented in format strings using %@ When an object appears in a format string, it is asked for its description
[NSString stringWithFormat: @The answer is: %@, myObject];
You can log an objects description with:
NSLog([anObject description]);
Your custom subclasses can override description to return more specific information
Total 67 Pages 23
Foundation Framework
Foundation Classes
NSObject
String
NSString / NSMutableString
Collection
NSArray / NSMutableArray NSDictionary / NSMutableDictionary NSSet / NSMutableSet NSNumber
Others
NSData / NSMutableData NSDate / NSCalendarDate
Total 67 Pages 24
More OOP Info
Tons of books and articles on OOP Objective-C 2.0 Programming Language
Total 67 Pages
25
Outlines
Objective-C basics Building an Application Sensor programming (accelerometer, GPS, microphone) Audio Libraries
Total 67 Pages
26
Anatomy of an Application
Compiled code
Your code Frameworks
Nib files
UI elements and other objects Details about object relations
Resources (images, sounds, strings, etc) Info.plist file (application configuration)
Total 67 Pages 27
App Lifecycle
Total 67 Pages
28
App Lifecycle
Main function UIApplicationMain which Info.plist to figure out what nib to load. MainWindow.xib contains the connections for our application. AppDelegate ViewController.xib View handle UI events
Total 67 Pages 29
Model-View-Controller
Total 67 Pages
30
Communication and MVC
Total 67 Pages
31
Model
Manages the app data and state Note concerned with UI or presentation Often persists somewhere Same model should be reusable, unchanged in different interfaces
Total 67 Pages
32
View
Present the Model to the user in an appropriate interface Allows user to manipulate data Does not store any data
(except to cache state)
Easily reusable & configurable to display different data
Total 67 Pages
33
Controller
Intermediary between Model & View Updates the view when the model changes Updates the model when the user manipulates the view Typically where the app logic lives
Total 67 Pages
34
Outlines
Objective-C basics Building an Application Sensor programming (accelerometer, GPS, microphone, ) Audio Libraries
Total 67 Pages
35
Accelerometer
What are the accelerometers?
Measure changes in force
What are the uses? Physical Orientation vs. Interface Orientation
Ex: Photos & Safari
Total 67 Pages
36
Orientation-Related Changes
Getting the physical orientation
UIDevice class
UIDeviceOrientationDidChangeNotification
Getting the interface orientation
UIApplication class
statusBarOrientation property
UIViewController class
interfaceOrientation property
Total 67 Pages
37
Shake Undo!
UIEvent type
@property(readonly) UIEventType type; @property(readonly) UIEventSubtype subtype; UIEventTypeMotion UIEventSubtypeMotionShake
Total 67 Pages
38
Getting raw Accelerometer Data
3-axis data Configurable update frequency (10-100Hz) Sample Code (AccelerometerGraph)
Class
UIAccelerometer UIAcceleration
Protocol
UIAccelerometerDelegate
Total 67 Pages 39
Getting raw Accelerometer Data
-(void)initAccelerometer{ [[UIAccelerometer sharedAccelerometer] setUpdateInterval: (1.0 / 100)]; [[UIAccelerometer sharedAccelerometer] setDelegate: self]; } -(void)accelerometer: (UIAccelerometer*)accelerometer didAccelerate: (UIAcceleration*) acceleration { double x, y, z; x = acceleration.x; y = acceleration.y; z = acceleration.z; //process the data } -(void)disconnectAccelerometer{ [[UIAccelerometer sharedAccelerometer] setDelegate: nil]; }
Total 67 Pages 40
Filtering Accelerometer Data
Low-pass filter
Isolates constant acceleration Used to find the device orientation
High-pass filter
Shows instantaneous movement only Used to identify user-initiated movement
Total 67 Pages
41
Filtering Accelerometer Data
Total 67 Pages
42
Filtering Accelerometer Data
Total 67 Pages
43
Applying Filters
Simple low-pass filter example
#define FILTERFACTOR 0.1 Value = (newAcceleration * FILTERFACTOR) + (previousValue * (1.0 FILTERFACTOR)); previousValue = value;
Simple high-pass filter example
lowpassValue = (newAcceleration * FILTERFACTOR) + (previousValue * (1.0 FILTERFACTOR)); previousLowPassValue = lowPassValue; highPassValue = newAcceleration lowPassValue;
Total 67 Pages
44
GPS
Classes
CLLocationManager CLLocation
Protocol
CLLocationManagerDelegate
Total 67 Pages 45
Getting a Location
Starting the location service
-(void)initLocationManager{ CLLocationManager* locManager = [[CLLocationManager alloc] init]; locManager.delegate = self; [locManager startUpdatingLocation]; }
Using the event data
-(void)locationManager: (CLLocationManager*)manager didUpdateToLocation: (CLLocation*)newLocation fromLocation: (CLLocation*)oldLocation{ NSTimerInterval howRecent = [newLocation.timestamp timeIntervalSinceNow]; if(howRecent < -10) return;
if(newLocation.horizontalAccuracy > 100) return; //Use the coordinate data. double lat = newLocation.coordinate.latitude; double lon = newLocation.coordinate.longitude; }
Total 67 Pages 46
Getting a Heading
Geographic North
CLLocationDirection trueHeading
Magnetic North
CLLocationDirection magneticHeading
-(void)locationManager: (CLLocationManager *)manager didUpdateHeading:(CLHeading*)newHeading{ //Use the coordinate data. CLLocationDirection heading = newHeading.trueHeading; CLLocationDirection magnetic = newHeading.magneticHeading; }
Total 67 Pages
47
Microphone
We will cover it in the audio library
Total 67 Pages
48
Outlines
Objective-C basics Building an Application Sensor programming (accelerometer, GPS, microphone) Audio Libraries
Total 67 Pages
49
Audio Libraries
System Sound API short sounds AVAudioPlayer ObjC, simple API Audio Session - Audio Toolbox Audio Queue - Audio Toolbox Audio Units OpenAL MediaPlayer Framework
Total 67 Pages
50
AVAudioPlayer
Play longer sounds (> 5 seconds) Locally stored files or in-memory (no network streaming) Can loop, seek, play, pause Provides metering Play multiple sounds simultaneously Cocoa-style API
Initialize with file URL or data Allows for delegate
Supports many more formats
Sample Code: avTouch
Everything the AudioFile API supports
Total 67 Pages
51
AVAudioPlayer
Create from file URL or data
AVAudioPlayer *player; NSString *path = [[NSBundle mainBundle] pathForResource: ofType:]; NSURL *url = [NSURL fileURLWithPath:path]; player = [[AVAudioPlayer allow] initWithContentsOfURL:url]; [player prepareToPlay];
Simple methods for starting/stopping
If(!player.playing){ [player play]; }else{ [player pause]; }
Total 67 Pages
52
Audio Sessions
Handles audio behavior at the application, inter-application, and device levels
Ring/Silent switch? iPod audio continue? Headset plug / unplug? Phone call coming?
Sample Code: avTouch & aurioTouch
Total 67 Pages
53
Audio Sessions
Six audio session categories
Ambient Solo Ambient (Default session) Media playback Recording Play and record Offline audio processing
Total 67 Pages
54
Audio Queue
Audio File Stream Services & Audio Queue Services Supports wider variety of formats Finer grained control over playback
Streaming audio over network Cf: AVAudioPlayer(local)
Allows queueing of consecutive buffers for seamless playback
Callback functions for reusing buffers
Sample code: SpeakHere
Total 67 Pages 55
Audio Units
For serious audio processing Graph-based audio
Rate or format conversion Real time input/output for recording & playback Mixing multiple streams VoIP (Voice over Internet Protocol).
Very, very powerful. Sample code: aurioTouch
Total 67 Pages 56
Audio Units
Lowest programming layer in iOS audio stack
Real-time playback of synthesized sounds Low-latency I/O Specific audio unit features
Otherwise, look first at the Media Player, AV Foundation, OpenAL, or Audio Toolbox!
Total 67 Pages
57
Audio Units
Ex: Karaoke app in iPhone real-time input from mic Real-time output to speaker Audio Unit provides excellent responsiveness Audio Unit controls audio flow to do pitch tracking, voice enhancement, iPod equalization, and etc.
Total 67 Pages
58
OpenAL
High level, cross-platform API for 3D audio mixing
Great for games Mimics OpenGL conventions
Models audio in 3D space
Buffers: Container for Audio Sources: 3D point emitting Audio Listener: Position where Sources are heard
Sample code: oalTouch
More information: http://www.openal.org/
Total 67 Pages 59
MediaPlayer Framework
Tell iPod app to play music Access to entire music library
For playback, not processing
Easy access through MPMediaPickerController Deeper access through Query APIs
Sample code: AddMusic
Total 67 Pages 60
Others
Accelerate Framework
C APIs for vector and matrix math, digital signal processing large number handling, and image processing vDSP programming guide
Bonjour and NSStream The Synthesis ToolKit in C++ (STK) and OSC
Total 67 Pages 61
More Info for Beginners
iPhone Application Programming (Standford University - iTunes University) Dan Pilone & Tracey Pilone. Head First iPhone Development.
Total 67 Pages
62
Thank you and good luck to your final projects!
Total 67 Pages
63
