Creating a Xamarin iOS Binding - WebRTC

Hello? Can you hear me?

Posted on 6 mins read

Recently I integrated video conferencing features into a Xamarin app using the open-source WebRTC framework. Google provides native libraries for both iOS and Android but I couldn’t find any recent Xamarin bindings. This left me with only one path forward, start at the beginning and create a binding from scratch.

Getting the Native Framework

Prerequisites

  1. Xamarin iOS development environment (i.e. macOS, XCode, VS for Mac, etc)
  2. CocoaPods (package manager for XCode)
  3. Objective Sharpie (tool to help getting our binding library started)

Create a blank iOS XCode project

Open XCode and create a plain old single view iOS app, nothing fancy. We’re just going to use this as a way to download the WebRTC CocoaPod that we’ll use to create our binding. You can uncheck the options for creating a UITest project and git repository.

Inside your newly created XCode project you should have the following directory structure,

WebRTC
WebRTC.xcodeproj

Initialise the CocoaPods Podfile

Open up a Terminal in the project directory and run the following,

pod init

Open the Podfile created in the project’s root and add in the reference to GoogleWebRTC,

platform :ios, '10.0'

target 'WebRTC' do  
  use_frameworks!

  pod 'GoogleWebRTC'
end

Install the Pod

In your Terminal window run the following,

$ pod install

Analyzing dependencies
Downloading dependencies
Installing GoogleWebRTC (1.1.31999)
Generating Pods project
Integrating client project
Pod installation complete! There is 1 dependency from the Podfile and 1 total pod installed.

Once that’s complete you should now have a folder called Pods in the project’s root. If you walk down that directory you’ll find a folder called WebRTC.framework, which is the native library that we’ll use in our binding.

Note: You can use Objective Sharpie to automatically create a binding from a CocoaPod, but I couldn’t get the feature to work.

Binding with Objective Sharpie

Now we’ll use Objective Sharpie to automatically generate the definitions to bind our Obj-C WebRTC library to C#.

Make a copy of WebRTC.framework so we can work on it safely. Open up a Terminal in the directory that contains the framework and run the below command,

sharpie bind -sdk iphoneos -output ./ -namespace Xam.WebRtc.iOS -scope ./WebRTC.framework/Headers ./WebRTC.framework/Headers/WebRTC.h
  • -sdk iphoneos create the binding for iPhone OS (i.e. iOS)
  • -output ./ put any resulting binding files in the current directory
  • -namespace Xam.WebRtc.iOS the C# namespace for the new binding
  • -scope ./WebRTC.framework/Headers ./WebRTC.framework/Headers/WebRTC.h limit the API scope to the header files of the framework, this stops Sharpie generating bindings for all referenced libraries in the framework (e.g. Foundation, UIKit, etc)

You’ll get the below error,

Parsing 1 header files...
/Users/gatto/Documents/WebRTC_Binding/WebRTC.framework/Headers/WebRTC.h:11:9: fatal error: 
      'WebRTC/RTCCodecSpecificInfo.h' file not found
#import <WebRTC/RTCCodecSpecificInfo.h>
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Binding...
1 error generated.
Error while processing /Users/gatto/Documents/WebRTC_Binding/WebRTC.framework/Headers/WebRTC.h.

Done. Exiting with error code 1.
error: Clang failed to parse input and exited with code 1

There is a problem with WebRTC.h, the umbrella header which imports all the necessary header files in one place.

Open it up and you’ll find it just one big list,

#import <WebRTC/RTCCodecSpecificInfo.h>
#import <WebRTC/RTCEncodedImage.h>
#import <WebRTC/RTCI420Buffer.h>
#import <WebRTC/RTCLogging.h>
...

Sharpie is complaining that it cannot find WebRTC/RTCCodecSpecificInfo.h which is the first import, it doesn’t know how to resolve the angle brackets (<>). Replace the brackets with quotes and remove "WebRTC/", as below, and rerun the command.

#import "RTCCodecSpecificInfo.h"
#import "RTCEncodedImage.h"
#import "RTCI420Bufferh"
#import "RTCLogging.h"
...

Sharpie should now have created two new files in the directory,

ApiDefinitions.cs
StructsAndEnums.cs

Creating the Binding Library

Create a Binding Project

Open up Visual Studio for Mac and start a new iOS Bindings Library.

Copy the WebRTC.framework directory into the project and add it as a Native Reference.

Next copy the contents of ApiDefinitions.cs and StructsAndEnums.cs generated by Sharpie into the corresponding files in the binding library.

Fixing the Binding Errors

The bindings generated by Sharpie are a great starting point, however, there are some things the tool cannot do which need to be fixed up manually.

214 Build Errors

The first place to start is with the Verify Hints. For this binding we’ll have to deal with MethodToProperty and ConstantsInterfaceAssociation.

Usually, MethodToPropery attributes are pretty safe to accept as is and can just be commented out. Essentially, Sharpie bound an Obj-C method to a C# property to expose a nicer API.

For this binding I didn’t need any of the constant fields bound in the interfaces marked with ConstantsInterfaceAssociation, so they were commented out.

[Static]
[Verify (ConstantsInterfaceAssociation)]
partial interface Constants
{
    ...
}

Next in a few classes there are some methods with the same signature, for example in RTCAudioSessionDelegate,

// @optional -(void)audioSession:(RTCAudioSession * _Nonnull)audioSession willSetActive:(BOOL)active;
[Export ("audioSession:willSetActive:")]
void AudioSession (RTCAudioSession audioSession, bool active);

// @optional -(void)audioSession:(RTCAudioSession * _Nonnull)audioSession didSetActive:(BOOL)active;
[Export ("audioSession:didSetActive:")]
void AudioSession (RTCAudioSession audioSession, bool active);

This is a pretty easy fix, we just need to give them a unique method name, which should be based on the export attribute,

// @optional -(void)audioSession:(RTCAudioSession * _Nonnull)audioSession willSetActive:(BOOL)active;
[Export ("audioSession:willSetActive:")]
void AudioSessionWillSetActive (RTCAudioSession audioSession, bool active);

// @optional -(void)audioSession:(RTCAudioSession * _Nonnull)audioSession didSetActive:(BOOL)active;
[Export ("audioSession:didSetActive:")]
void AudioSessionDidSetActive (RTCAudioSession audioSession, bool active);

There is a random dispatch_block_t type used as a parameter on one the function definitions. To fix this we need to create a delegate to handle the call back.

// +(void)dispatchAsyncOnType:(RTCDispatcherQueueType)dispatchType block:(dispatch_block_t)block;
[Static]
[Export("dispatchAsyncOnType:block:")]
void DispatchAsyncOnType(RTCDispatcherQueueType dispatchType, dispatch_block_t block);

// Becomes...

delegate void NSDispatchHandler();

// +(void)dispatchAsyncOnType:(RTCDispatcherQueueType)dispatchType block:(dispatch_block_t)block;
[Static]
[Export("dispatchAsyncOnType:block:")]
void DispatchAsyncOnType(RTCDispatcherQueueType dispatchType, NSDispatchHandler block);

There are a few errors relating to CVPixelBufferRef*, for which Visual Studio cannot find the type. The *Ref types in Obj-C are pointers to the actual object, and since C# is managed it doesn’t bind correctly. You can just remove the Ref* bit at the end and remove the unsafe keyword.

// @property (readonly, nonatomic) CVPixelBufferRef _Nonnull pixelBuffer;
[Export ("pixelBuffer")]
unsafe CVPixelBufferRef* PixelBuffer { get; }

// Becomes...

// @property (readonly, nonatomic) CVPixelBufferRef _Nonnull pixelBuffer;
[Export ("pixelBuffer")]
CVPixelBuffer PixelBuffer { get; }

Next the errors related to the byte* type, which is a pointer to a byte array. The proper way to expose a friendly C# API for these fields is to write some code that’ll Marshall the memory into, and out of, the managed .NET world. However, I neither have the skill, nor the requirement to access these fields at the moment, so it is safe to change them all to the IntPtr type.

// @required @property (readonly, nonatomic) const uint8_t * _Nonnull dataY;
[Abstract]
[Export ("dataY")]
unsafe byte* DataY { get; }

// Becomes...

// @required @property (readonly, nonatomic) const uint8_t * _Nonnull dataY;
[Abstract]
[Export ("dataY")]
unsafe IntPtr DataY { get; }

Finally, we’ll add a few interface definitions to map the Obj-C protocols. For example,


public interface IRTCPeerConnectionDelegate { }

// @protocol RTCPeerConnectionDelegate <NSObject>
[Protocol, Model(AutoGeneratedName = true)]
[BaseType(typeof(NSObject))]
interface RTCPeerConnectionDelegate
{
    ...
}

// @interface RTCPeerConnectionFactory : NSObject
[BaseType(typeof(NSObject))]
 RTCPeerConnectionFactory
{
    ...
    // -(RTCPeerConnection * _Nonnull)peerConnectionWithConfiguration:(RTCConfiguration * _Nonnull)configuration constraints:(RTCMediaConstraints * _Nonnull)constraints delegate:(id<RTCPeerConnectionDelegate> _Nullable)delegate;
    [Export("peerConnectionWithConfiguration:constraints:delegate:")]
    RTCPeerConnection PeerConnectionWithConfiguration(RTCConfiguration configuration, RTCMediaConstraints constraints, [NullAllowed] IRTCPeerConnectionDelegate @delegate);
    ...
}

Wrapping Up

Once the binding was building, I created a sample project based on some native Swift examples I found, essentially porting the code over to C#. This might not be the cleanest C# API and is probably missing some functionality, especially if you want to modify/inspect the raw video buffer, but it suits my requirements. Find the all the code and samples here.

Happy for pull requests and advise on how to make this a better binding!

Resources

Update March 2021

You should also read the awesome binding tutorials for WebRTC by Paula Aliu, who does a walkthrough of both iOS and Android, also available on GitHub!

comments powered by Disqus