In collaboration with succesful Dutch game studio ‘Knuist & Perzik’, the creators of indie game Wuppo, we’ve released the minigame Fnakball on the AirConsole platform. The game can be played here.
It’s a fun little multiplayer minigame, where you play a game volleyball, only with a stick. There are no rules, you just need to make sure the ball hits the ground on your opponent’s side of the field.
AirConsole is a platform that runs in either the browser, or on an Android TV, and uses your phone as a controller. The main game itself was built in Unity, but the controllers need to be hosted as HTML pages, with functionality implemented in Javascript. This allowed us to implement a complete UI on the controller. You customise your character on your phone screen, vote for a level, change your own user settings, and control the game. For in-game controls, you tilt your phone to move your character, tap on the left side of the screen to jump, and either tap or swipe on the right side of the screen to swing your stick and hit the ball. This allows you to play the game without having to look at your phone.
Having the controls on your phone did give me some interesting challenges. The game and the phone communicate over a WebRTC connection, so you’re constantly sending messages from one to the other to keep the game-state in sync. In addition, I was constantly switching between two codebases, namely the game in Unity C# and the controller in Javascript/Typescript. I solved both of these problems by building a small custom framework.
I made it so you could build the entire UI of the controller in C#, and even interact with it as if it were running locally. To create a scene in C# that runs on the phone, you create a class that inherits from ‘PhoneScene’, you define the visual properties, and construct the scene in the constructor. You can even subscribe to button pressed events here, and handle logic directly in C#.
Scaling the UI to different phone sizes and aspect ratios is also handled in this system. You can define the entire UI in a resolution of 1920 by 1080, define how every object should scale and move with different screen sizes, and then the system will handle the rest when rendering.
public class StageSelectScene : PhoneScene
{
private Image background;
private Image stageImage;
private Button cancelButton;
private Button voteButton;
public StageSelectScene()
{
background = new Image("TeamSelect/background.png");
background.ScaleMode = ScreenScaleMode.Biggest;
voteButton = new Button("StageSelect/vote_btn.png");
voteButton.Position = new Vector2(global::Player.Size.x - 547.0f, global::Player.Size.y - 313.0f);
voteButton.Anchor = AnchorPoint.Bottom | AnchorPoint.Right;
voteButton.OnPressed += (sender, args) =>
{
if(valid)
{
voteButton.Source = "StageSelect/voted_btn.png";
voteButton.Enabled = false;
Vote = currentStage;
(ScreenManager.Instance.ActiveScene as global::StageSelectScene)?.PlayerVoted();
}
else
{
Player.GetPremium();
}
};
The way this works is by making use of Reflection in C#. When the scene is entered, the class is instantiated and all visual properties are serialized into a JSON structure, which is sent to the phone. There in Typescript, it is deserialized, and by making use of Javascript’s unique dynamic features, the properties are added dynamically to the Scene object.
FieldInfo[] fields = GetType().GetFields(BindingFlags.Instance | BindingFlags.NonPublic);
foreach(var field in fields)
{
if(field.FieldType.IsSubclassOf(typeof(DisplayObject)))
{
DisplayObject obj = (DisplayObject)field.GetValue(this);
obj.OnPropertyChanged = PropertyChanged;
objects.Add(field.Name, obj);
}
else if(typeof(IList).IsAssignableFrom(field.FieldType))
{
IList listObject = (IList)field.GetValue(this);
if(listObject.GetType().GetGenericArguments()[0].IsSubclassOf(typeof(DisplayObject)))
{
foreach (var obj in listObject)
{
(obj as DisplayObject).OnPropertyChanged = PropertyChanged;
}
objects.Add(field.Name, listObject);
}
}
}
public Create(data:string)
{
let objData = JSON.parse(data);
let me = this as any;
for(let key in objData)
{
if(objData[key] instanceof Array)
{
me[key] = [];
for(let obj of objData[key])
{
let displayObj = new (AirGUI as any)[obj.TypeName](obj);
me[key].push(displayObj);
}
}
else
{
if(objData[key].TypeName) me[key] = new (AirGUI as any)[objData[key].TypeName](objData[key]);
else me[key] = objData[key];
}
}
}
The entire scene is then rendered on the phone on an HTML5 Canvas. Whenever a button is pressed on the phone, it is simply sent as a message to Unity, where the ButtonPressed event handler is then called on the appropriate Button in C#.
if(element instanceof Button)
{
if(element.TouchUpInside)
{
this.CallFunction("ButtonPressed", { "buttonName": key });
}
}
protected CallFunction(functionName:string, args:any = {})
{
window.airConsole.message(AirConsole.SCREEN, { "cmd": "CallFunction", "function": functionName, "args": args });
}
public void ButtonPressed(JToken args)
{
string buttonName = (string)args["buttonName"];
FieldInfo field = GetType().GetField(buttonName, BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Public);
if(field == null) return;
if(typeof(Button).IsAssignableFrom(field.FieldType))
{
var button = (Button)field.GetValue(this);
button.Pressed();
}
else if (typeof(IList).IsAssignableFrom(field.FieldType))
{
int arrayIndex = (int)args["arrayIndex"];
var listObject = (IList)field.GetValue(this);
((Button)listObject[arrayIndex]).Pressed();
}
}
I’ve uploaded the relevant parts of this UI system, which I’ve called AirGUI, in case you’re interested.
I think this system could quite easily be expanded to use the Unity Canvas, so you’d have a visual editor for your UI, but that was sadly out of scope for our project. It did however make it very easy for us to implement the entire control scheme, without having to worry about constantly syncing state between the phone and the game itself.