iOS におけるサウンド処理2015

iOSにおけるサウンド処理2015

• me.name = “Tamiya Tokihiko”

• me.company = “Classmethod, Inc.”

• company.blog = “Developers.IO”

自己紹介（プロパティ）

概要• AVAudioEngine

• iOS 8 から加わったクラス

• Audio Unit Extension

• iOS 9 で追加されたExtension Point

• Audio Unit をアプリ間で共有できる

• AVAudioSessionModeSpokenAudio

• 声コンテンツが他アプリの音声の妨げにならない仕組み

AVAudioEngine

このままだと不便

© Apple

© Apple

AVFoudation が拡充

© Apple

© Apple

AVAudioEngine

AVAudioPlayerNode

AVAudioUnit Delay

AVAudioMixerNode

AVAudio OutputNodeAVAudioFile

@property (nonatomic, strong) AVAudioEngine *engine;

@property (nonatomic, strong) AVAudioPlayerNode *audioPlayerNode;

@property (nonatomic, strong) AVAudioFile *audioFile;

@property (nonatomic, strong) AVAudioUnitDelay *audioUnitDelay;

プロパティ

self.engine = [AVAudioEngine new]; NSString *path = [[NSBundle mainBundle] pathForResource:@"loop.m4a" ofType:nil];

self.audioFile = [[AVAudioFile alloc] initForReading:[NSURL fileURLWithPath:path] error:nil];

AVAudioEngine と AVAudioFile

AVAudioEngine

self.audioPlayerNode = [AVAudioPlayerNode new];

[self.engine attachNode:self.audioPlayerNode];

AVAudioPlayerNode

AVAudioEngine

AVAudioPlayerNode

self.audioUnitDelay = [AVAudioUnitDelay new];

self.audioUnitDelay.wetDryMix = 50;

[self.engine attachNode:self.audioUnitDelay];

AVAudioEngine

AVAudioPlayerNode

AVAudioUnit Delay

AVAudioUnitDelay

AVAudioMixerNode *mixerNode = [self.engine mainMixerNode];

AVAudioEngine

AVAudioPlayerNode

AVAudioUnit Delay

AVAudioMixerNode

AVAudioMixerNode を生成

[self.engine connect:self.audioPlayerNode to:self.audioUnitDelay format:self.audioFile.processingFormat];

AVAudioEngine

AVAudioPlayerNode

AVAudioUnit Delay

AVAudioMixerNode

ノード同士をつなぐ

[self.engine connect:self.audioUnitDelay to:mixerNode format:self.audioFile.processingFormat];

AVAudioEngine

AVAudioPlayerNode

AVAudioUnit Delay

AVAudioMixerNode

ノード同士をつなぐ

NSError *error;

[self.engine startAndReturnError:&error];

AVAudioEngine

AVAudioPlayerNode

AVAudioUnit Delay

AVAudioMixerNode

AVAudioPlayerNodeで 音を再生

AVAudioEngine

AVAudioPlayerNode

AVAudioUnit Delay

AVAudioMixerNode

[self.audioPlayerNode scheduleFile:self.audioFile atTime:nil completionHandler:^() {}];

AVAudio OutputNodeAVAudioFile

/*! @property delayTime Time taken by the delayed input signal to reach the output @abstract Range: 0 -> 2 Default: 1 Unit: Seconds */ @property (nonatomic) NSTimeInterval delayTime;

/*! @property feedback @abstract Amount of the output signal fed back into the delay line Range: -100 -> 100 Default: 50 Unit: Percent */ @property (nonatomic) float feedback;

AVAudioUnitDelay.h

/*! @property lowPassCutoff @abstract Cutoff frequency above which high frequency content is rolled off Range: 10 -> (samplerate/2) Default: 15000 Unit: Hertz */ @property (nonatomic) float lowPassCutoff;

/*! @property wetDryMix @abstract Blend of the wet and dry signals Range: 0 (all dry) -> 100 (all wet) Default: 100 Unit: Percent */ @property (nonatomic) float wetDryMix;

AVAudioUnitDelay.h

demo

AudioUnitExtension

© Apple

AVAudioPlayerNode

AVAudioUnit Delay

AVAudioMixerNode

AVAudio OutputNodeAVAudioFile

@property (nonatomic) AVAudioEngine *engine;

@property (nonatomic) AVAudioPlayerNode *audioPlayerNode;

@property (nonatomic) AVAudioFile *file;

@property (nonatomic) AUAudioUnit *audioUnit;

@property (nonatomic) AVAudioUnit *effect;

@property (nonatomic) NSArray *items;

プロパティ

self.audioPlayerNode = [AVAudioPlayerNode new];

self.engine = [AVAudioEngine new];

self.effect = [AVAudioUnit new];

[self.engine attachNode:self.audioPlayerNode];

[self.engine connect:self.audioPlayerNode to:self.engine.mainMixerNode format:self.file.processingFormat];

ノード同士をつなぐ

AVAudioUnitComponent が入った配列を取得

AudioComponentDescription anyEffectDescription;

anyEffectDescription.componentType = kAudioUnitType_Effect;

anyEffectDescription.componentSubType = 0; anyEffectDescription.componentManufacturer = 0; anyEffectDescription.componentFlags = 0; anyEffectDescription.componentFlagsMask = 0; self.items = [[AVAudioUnitComponentManager sharedAudioUnitComponentManager] componentsMatchingDescription:anyEffectDescription];

- (void)viewWillAppear:(BOOL)animated { [super viewWillAppear:animated]; [self.engine startAndReturnError:nil]; [self scheduleLoop]; [self.audioPlayerNode play]; }

- (void)scheduleLoop { [self.audioPlayerNode scheduleFile:self.file atTime:nil completionHandler:^{ [self scheduleLoop]; }]; }

AVAudioPlayerNode 再生

[self.audioPlayerNode pause]; [self.engine disconnectNodeInput:self.effect]; [self.engine disconnectNodeInput:self.engine.mainMixerNode]; [self.engine connect:self.audioPlayerNode to:self.engine.mainMixerNode format:self.file.processingFormat]; self.effect = nil; self.audioUnit = nil;

ノードをつなぎ直す

self.effect = audioUnit;

（略）

[self.engine connect:self.audioPlayerNode to:self.effect format:self.file.processingFormat];

[self.engine connect:self.effect to:self.engine.mainMixerNode format:self.file.processingFormat];

特定のエフェクトを実現する AVAudioUnit を取得

[AVAudioUnit instantiateWithComponentDescription:componentDescription options:kAudioComponentInstantiation_LoadOutOfProcess completionHandler:^(__kindof AVAudioUnit * _Nullable audioUnit, NSError * _Nullable error) {}];

AVAudioUnit を使いノードをつなぐ

AVAudioUnitComponent *auComponent 　= self.items[indexPath.row - 1];

auComponent.name;

auComponent.manufacturerName;

AVAudioUnitComponent の名前・製造元

auComponent = self.items[indexPath.row - 1];

[self selectEffectWithComponentDescription:auComponent.audioComponentDescription];

[self.audioUnit requestViewControllerWithCompletionHandler:^(AUViewController * _Nullable viewController) {}];

独自の View を持っている場合 取得

demo

AVAudioSessionMode SpokenAudio

声のコンテンツでありがちな問題

聞き逃した！

Podcast に切替え

巻き戻し早送り 再生

AVAudioSessionCategoryOptionInterrupt SpokenAudioAndMixWithOthers

AVAudioSessionModeSpokenAudio

一時停止 ちょっと巻き戻し

概要• AVAudioEngine

• iOS 8 から加わったクラス

• Audio Unit Extension

• iOS 9 で追加されたExtension Point

• Audio Unit をアプリ間で共有できる

• AVAudioSessionModeSpokenAudio

• 声コンテンツが他アプリの音声の妨げにならない仕組み