Programmatic AutoLayout Constraints Basics for Xamarin

  1. Create element without an explicit Frame.
  2. Set TranslatesAutoresizingMaskIntroConstraints = false
  3. Create an array of NSLayoutConstraints
  4. Work top-to-bottom, left-to-right, or vice versa. Do this consistently throughout program
  5. Use Layout Anchors
  6. Use the top-level UIViews SafeAreaLayoutGuide to position relative to the Window / screen
  7. For each dimension, set its location (LeadingAnchor / TopAnchor or TrailingAnchor / BottomAnchor)
  8. Either set the other location anchor or set the internal dimension (WidthAnchor / HeightAnchor)
  9. Call NSLayoutConstraint.ActivateConstraints after the UIView and any referenced UIView objects have been added to the View Hierarchy (compiles OK, but runtime exception)
toolbar = new UIToolbar();
toolbar.TranslatesAutoresizingMaskIntoConstraints = false;
var tbConstraints = new[]
{
toolbar.LeadingAnchor.ConstraintEqualTo(this.View.SafeAreaLayoutGuide.LeadingAnchor),
toolbar.TrailingAnchor.ConstraintEqualTo(this.View.SafeAreaLayoutGuide.TrailingAnchor),
toolbar.TopAnchor.ConstraintEqualTo(this.View.SafeAreaLayoutGuide.TopAnchor),
toolbar.HeightAnchor.ConstraintEqualTo(toolbar.IntrinsicContentSize.Height)
};
View.AddSubview(toolbar);
NSLayoutConstraint.ActivateConstraints(tbConstraints);

label = new UILabel();
label.Text = "This is the detail view";
label.TranslatesAutoresizingMaskIntoConstraints = false;
var lblConstraints = new[]
{
label.LeadingAnchor.ConstraintEqualTo(this.View.SafeAreaLayoutGuide.LeadingAnchor, 20.0f),
label.WidthAnchor.ConstraintEqualTo(label.IntrinsicContentSize.Width),
label.TopAnchor.ConstraintEqualTo(this.toolbar.BottomAnchor, 20.0f),
label.HeightAnchor.ConstraintEqualTo(label.IntrinsicContentSize.Height)
};
View.AddSubview(label);
NSLayoutConstraint.ActivateConstraints(lblConstraints);

Notes on installing TensorFlow with GPU Support

The best Tensorflow is the one you have on your machine.

In my opinion, the bottleneck on a DNN solution is not training, but data preparation and iterating your model to the point where it’s reasonable to start investing kilowatt-hours of electricity to the training. So I have Tensorflow on all my machines, including my Macs, even though as of Tensorflow 1.2 GPU support is simply not available for Tensorflow on the Mac. (I’m not sure what’s going on, but suspect it may have something to do with licensing NVidia’s CuDNN library.)

Having said that, GPU support for TensorFlow is much faster than CPU-only Tensorflow (in some quick tests on my Windows laptops, ~8x). With GPU-supported Tensorflow, it’s that much easier to iterate your model until your training and validation curves start to look encouraging. At that point, in my opinion it makes sense to move your training to the cloud. There’s a little more friction in terms of moving data and starting and stopping runs and you’re paying for processing, but hopefully you’ve gotten to the point where training time is the bottleneck.

Anyway…

Mac Tensorflow GPU: I’d like to think this will change in the future, but as of August 2017: Nope.

There are a very few people who seem to have figured out how to build Tensorflow with GPU support on the Mac from sources, but the hoop-jumping and yak shaving that seems necessary seems very high to me.

Windows Tensorflow GPU: Yes, but it’s a little finicky. Here are some install notes:

– Install NVidia Cuda 8 (not the Cuda 9 RC)
– Install NVidia CuDNN 5.1 (not the CuDNN 7!)
– Copy the CuDNN .dll to your Cuda /bin directory (probably /Program Files/NVidia GPU Computing Toolkit/Cuda/v8.0/bin/)
– Create an Anaconda environment from an administrative shell. Important: use –python=3.5
– Install tensorflow using:

pip install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/windows/gpu/tensorflow_gpu-1.2.1-cp35-cp35m-win_amd64.whl

I think the “cp35” is the hint that you have to use Python 3.5, so if the page at https://www.tensorflow.org/install/install_windows changes to show a different .whl file, you’d have to set the python in your Anaconda environment differently.
– Validate that you’ve got GPU capability:

import tensorflow as tf
tf.Session().run(tf.constant('hello'))

This should result in a cascade of messages, many of which say that Tensorflow wasn’t compiled with various CPU instructions, but most importantly, towards the end you should see a message that begins:

Creating Tensorflow device (/gpu:0)

which indicates that, sure enough, Tensorflow is going to run on your GPU.

 

Hope this helps!

Posted in AI

Dell Infuriates Me

Sunday rant: I have a 2016 Dell XPS (high-end) laptop. I bought a Dell 25″ 4K monitor. And on Friday received a Dell Thunderbolt dock for the monitor. I plug it all together and although the monitor displays wonderfully, the dock is not passing USB through. So I start fiddling around with “unplug from dock, plug into laptop, confirm the peripheral is working,” stuff. And then the laptop BSODs. Machine boots, connects to dock, everything’s fine for 30 seconds, BSOD. Start to Google. “Update your laptop BIOS.” (For a fucking docking station!). It takes a goddamn hour to find the BIOS update on the Dell Website for their goddamn flagship laptop, but whatever.

Still BSODs. Now it’s telling me that I have to update the firmware on the dock. But I cannot update the firmware because if I attach the dock to the laptop to update it, it BSODs. So, there’s this few-second window before the BSOD where I see that I have to update my Thunderbolt Driver on the laptop.

So I download the driver and run the installer for the Thunderbolt Driver. The installer doesn’t give any option other than “Uninstall.” So I say “OK, I’ll uninstall and reinstall.” I uninstall. Fine. I go to reinstall. I’m told I don’t have sufficient permission. So I run as administrator. I still don’t have sufficient permission. So I end up editing the registery to turn off user protection. (Remember, this is all for a docking station).

I now can run the “install” option, but it refuses to continue because it sees some pre-existing value in the registry. (Which I take to mean it’s “Uninstall” function didn’t actually, you know, uninstall.) It then rolls back the Thunderbolt install and leaves me with my current situation:

A half-upgraded machine with user access protection turned off, less functionality than it had before, and it still BSODs whenever I turn on the dock. All with a respected company’s flagship hardware.

God.

Programmed my first Alexa skill: I was shocked by what I found!

Although I am pretty deeply entrenched in the Apple ecosystem, the recently-announced $50 Dot was so inexpensive I could not resist checking it out. (Before I go further: I work for Microsoft, so take that into account as you see fit.)

Out of the box, the Echo is very easy to setup for basic queries “Alexa, what’s my latitude and longitude?” and so forth. The Echo has a relatively lo-fi speaker and the integration with Sonos (what Amazon calls an “Alexa Skill”) is not yet available, so I haven’t used it all that much.

But there’s an API so you know I had to program something. My preferred solution for “computations in the cloud” is definitely Azure Functions written in F#, but for my first Alexa Skill I used Amazon Lambda running Python.

The first thing to focus on is that Alexa Skills are a separate service that can be programmed many ways, so there’s always going to be a certain amount of integration overhead in the form of multiple tabs open, jumping back and forth between the Alexa Skills and the Web server/service where you are handling the computation.

The Alexa Skills documentation is good, but there’s a good number of parts and I think it’s wise to write your first skill using Amazon Lambda, as I did. Amazon Lambda is often the default service in the documentation and there are often hyperlinks to the Lambda-specific page to do “X.”


A Skill for Gravity

A friend was talking to me about riflery and astonishing me with the flight times he was talking about. Alexa failed to answer some basic questions about ballistics (Alexa seems to me less capable than Google Assistant, Cortana, or Siri at answering freeform questions), offering me the perfect simple use-case for my first skill.

Minimum viable query: "What is the speed of an object that has fallen for 1.5 seconds?"

SWAG achievable: "How long would it take for an object dropped from the height of the Empire State Building to fall to the ground on Mars?"

The nice thing about my minimal query is that it’s both stateless and easy to answer with some math: all you need to answer is the duration of the drop and use a gravitational constant of -9.81. (Conversions from meters/second can come later.)

I followed the documentation on building an Alexa skill with a Lambda function to create an Alexa Skill named called “Gravity.” After naming, the next page of the Skill development site is “Interaction Model.” This is where I was shocked to discover:

Alexa doesn’t do natural language processing!

I ASS-U-ME’d that I would be receiving some programmatic structure that told me the “nominal subject” of the sentence was the noun speed and would allow me to search for a “prepositional modifier” whose “object” was the noun seconds and extract its modifier. That would allow me to recognize either of these sentences:

  • What is the speed of an object that has fallen for 1.5 seconds?; or
  • What's the velocity of an apple after after 1.5 seconds?

Or any of a large number of other sentences. Foxtype will show you such parsing in action at this (fascinating) page.

But no! As you can see in the screenshot below, the mapping of a recognized sentence to a programmatic “intent” is nothing but a string template! You either have to anticipate every single supported structure or you have to use wildcards and roll your own. (Honestly, I imagine that it’s not a long road before the wisest interaction model is Parse {utterance}.)

intents1

To be clear: ‘just’ voice recognition is extraordinarily hard and doing it in ambient environmental noise is insane. It’s only because Alexa already does this very, very hard task that it’s surprising to me that they don’t provide for some amount of the (also hard) task of parsing. The upside, of course, is that sound->utterance is decoupled from utterance->sentence. As far as I know, no one today provides “NLP as a Service” but it’s easy to imagine. (Although latency… Nope, nope, staying on topic…)

Returning to the screenshot above, you can see that it contains the bracketed template {duration}. The matching value will be associated with the key duration in calls to the Lambda function. And, to be honest, it’s a place where Alexa Kit does do some NLP.

You can help Alexa by specifying the type of the variables in your template text. For instance, I specified the duration variable as a NUMBER. Alexa does use NLP to transform the utterances meaningfully — so “one and a half” becomes “1.5” and so forth. I haven’t really explored the extent of this — does it turn “the Tuesday after New Year’s Day” into a well-formed date and so forth?

Alexa packages session data relating to an ongoing conversation and intent data and performs an RPC-like call (I actually don’t know the details) to the endpoint of your choice. In the case of Amazon Lambda, that’s the Amazon Resource Name (ARN) of your function.

The data structures it passes look like this:

[code lang=”javascript”]
{
"session": {
"sessionId": "SessionId.07dc1151-eb4e-4e12-98fa-64af3f59d82a",
"application": {
"applicationId": "amzn1.ask.skill.443f7cb5-ETC-dbecb288ff2d"
},
"attributes": {},
"user": {
"userId": "amzn1.ask.account.ETC"
},
"new": true
},
"request": {
"type": "IntentRequest",
"requestId": "EdwRequestId.13cf7a2b-0789-4244-879f-f4fae08f315f",
"locale": "en-US",
"timestamp": "2016-11-18T17:24:09Z",
"intent": {
"name": "FallingSpeedIntent",
"slots": {
"duration": {
"name": "duration",
"value": "1.5"
}
}
}
},
"version": "1.0"
}
[/code]

The values in the session object relate to a conversation and the values in the request object belong to a specific intent — in this case the FallingSpeedIntent with the duration argument set to “1.5”.

On the Lambda side of things

Amazon Lambda has a template function called ColorIs that provides an easy starting point. It supports session data, which my Gravity skill doesn’t require, so I actually ended up mostly deleting code (always my favorite thing). Given the JSON above, here’s how I route the request to a specific function:

[code lang=”python”]
def on_intent(intent_request, session):
""" Called when the user specifies an intent for this skill """

print("on_intent requestId=" + intent_request[‘requestId’] +
", sessionId=" + session[‘sessionId’])

intent = intent_request[‘intent’]
intent_name = intent_request[‘intent’][‘name’]

# Dispatch to your skill’s intent handlers
if intent_name == "FallingSpeedIntent" :
return get_falling_speed(intent, session)

def get_falling_speed(intent, session):
session_attributes = {}
reprompt_text = None
should_end_session = True

g = -9.82 #meters per second squared

if "duration" in intent[‘slots’]:
duration = float(intent[‘slots’][‘duration’][‘value’])
velocity = g * duration**2

speech_output = "At the end of " + str(duration) + " seconds, an object will be falling at " + (‘%.1f’ % velocity) + " meters per second. " + \
"Goodbye."
else:
speech_output = "Pretty fast I guess."

return build_response(session_attributes, build_speechlet_response(
intent[‘name’], speech_output, reprompt_text, should_end_session))

[/code]

(Boilerplate not shown)

My Westworld prediction

[code lang=”csharp”]
var k = Convert.FromBase64String("vlqnRQo8YYXdqt3c7CahDninF6MgvRnqNEU+/tcbWdM=");
var iv = Convert.FromBase64String("gaXwv734Tu3+Jw1hgtNrzw==");
DecryptStringFromBytes(Convert.FromBase64String("Yr2XWzCxceStAF1BaUgaqmWcqFjzWskDDN4foaxfGEO5JHc/oKvgukkMHZuOiw+dK0JxnOhzC1ZA3QLqZZsQxFtjX+qvu0VRM0p6VEfcv18="), k, iv);[/code]

Keras is the Deep Learning Toolkit You Have Been Waiting For

I remember a time when “learning about computers” invariably started with the phrase “computers only operate on 0s and 1s…” Things could vary a little for a few minutes, but then you’d get to the meat of things: Boolean logic. “All computer programs are formed from these ‘logic gates’…”

I remember a poster that illustrated Boolean logic in terms of punching. A circuit consisted of a bunch of mechanical fists, an “AND” gate propagated the punch when both its input were punched, an “OR” required only one input punch, etc. At the bottom were some complex circuits and the ominous question: “Are you going to be punched?” Because Boston. (The answer was “Yes. You are going to be punched.”)

Anyway, the point is that while there was a fundamental truth to what I was being told, it was not overwhelmingly relevant to the opportunities that were blossoming, back then at the dawn of the personal computer revolution. Yes, it’s important to eventually understand gates and circuits and transistors and yes, there’s a truth that “this is all computers do,” but that understanding was not immediately necessary to get cool results, such as endlessly printing “Help, I am caught in a program loop!” or playing Nim or Hammurabi. Those things required simply typing in a page or two of BASIC code.

Transcription being what it is, you’d make mistakes and curiosity being what it is, you’d mess around to see what you could alter to customize the game, and then your ambition would slowly grow and only then would you start to benefit from understanding the foundations on which you were building.

Which brings us to deep learning.

You have undoubtedly noticed the rising tide of AI-related news involving “deep neural nets.” Speech synthesis, Deep Dream’s hallucinogenic dog-slugs, and perhaps most impressively AlphaGo’s success against the 9-dan Lee Sedol. Unlike robotics and autonomous vehicles and the like, this is purely software-based: this is our territory.

But “learning about deep learning” invariably starts with phrases involving the phrases “regression,” “linearly inseparable,” and “gradient descent.” It gets math-y pretty quickly.

Now, just as “it’s all just 0s and 1s” is both true but not immediately necessary, “it’s all just weights and transfer functions,” is something for which_eventually_ you will want to have an intuition. But the breakthroughs in recent years have not come about so much because of advances at this foundational level, but rather from a dramatic increase in sophistication about how neural networks are “shaped.”

Not long ago, the most common structure for an artificial neural network was an input layer with a number of neural “nodes” equal to the number of inputs, an output layer with a node per output value, and a single intermediate layer. The “deep” in “deep learning” is nothing more than networks that have more than a single intermediate layer!

Another major area of advancement is approaches that are more complex than “an input node equal to the number of inputs.” Recurrence, convolution, attention… all of these terms relate to this idea of the “shape” of the neural net and the manner in which inputs and intermediate terms are handled.

… snip descent into rabbit-hole …

The Keras library allows you to work at this higher level of abstraction, while running on top of either Theano or TensorFlow, lower-level libraries that provide high-performance implementations of the math-y stuff. This is a Keras description of a neural network that can solve the XOR logic gate. (“You will get punched if one, but not both of the input faces gets punched.”)

[code lang=”python”]
import numpy as np
from keras.models import Sequential
from keras.layers.core import Activation, Dense
from keras.optimizers import SGD

X = np.zeros((4, 2), dtype=’uint8′)
y = np.zeros(4, dtype=’uint8′)

X[0] = [0, 0]
y[0] = 0
X[1] = [0, 1]
y[1] = 1
X[2] = [1, 0]
y[2] = 1
X[3] = [1, 1]
y[3] = 0

model = Sequential()
model.add(Dense(2, input_dim=2))
model.add(Activation(‘sigmoid’))
model.add(Dense(1))
model.add(Activation(‘sigmoid’))

sgd = SGD(lr=0.1, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss=’mean_squared_error’, optimizer=sgd, class_mode="binary")

history = model.fit(X, y, nb_epoch=10000, batch_size=4, show_accuracy=True, verbose=0)

print (model.predict(X))
[/code]

I’m not claiming that this should be crystal clear to a newcomer, but I do contend that it’s pretty dang approachable. If you wanted to produce a different logic gate, you could certainly figure out what lines to change. If someone told you “The ReLu activation function is used more often than sigmoid nowadays,” your most likely ‘let me see if this works’ would, in fact, work (as long as you guessed you should stick with lowercase).

For historical reasons, solving XOR is pretty much the “Hello, World!” of neural nets. It can be done with relatively little code in any neural network library and can be done in a few dozen lines of mainstream programming languages (my first published article was a neural network in about 100 lines of C++. That was… a long time ago…).

But Keras is not at all restricted to toy problems. Not at all. Check this out. Or this. Keras provides the appropriate abstraction level for everything from introductory to research-level explorations.

Now, is it necessary for workaday developers to become familiar with deep learning? I think the honest answer to that is “not yet.” There’s still a very large gap between “what neural nets do well” and “what use-cases are the average developer being asked to addressed?”

But I think that may change in a surprisingly short amount of time. In broad terms, what artificial neural nets do is recognize patterns in noisy signals. If you have a super-clean signal, traditional programming with those binary gates works. More importantly, lots of problems don’t seem easily cast into “recognizing a pattern in a signal.” But part of what’s happening in the field of deep learning is very rapid development of techniques and patterns for re-casting problems in just this way. So-called “sequence-to-sequence” problems such as language translation are beginning to rapidly fall to the surprisingly effective techniques of deep learning.

… snip descent into rabbit-hole …

Lots of problems and sub-problems can be described in terms of “sequence-to-sequence.” The synergy between memory, attention, and sequence-to-sequence — all areas of rapid advancement — is tipping-point stuff. This is the stuff of which symbolic processing is made. When that happens, we’re talking about real “artificial intelligence.” Artifical intelligence, yes, but not, I think, human-level cognition. I strongly suspect that human-level, general-purpose AI will have a trajectory similar to medicine based on genetics: more complex and messy and tangled to be cracked with a single breakthrough.

Debugging provisioning profiles on the command line

Raise your hand if you’ve ever struggled with getting your app’s bundle identifier, info.plist, and entitlements.plist to match up with your provisioning profile.

I tried to explain provisioning profiles using the ten-hundred most common words, but in slightly-less-common words, a development prov-pro associates: A team, a developer, an application identifier, privacy and security entitlements, and development devices.

While there’s no silver bullet, there is a way to dump the contents of a provisioning profile into a readable plist format. From the command-line, run:

security cms -D -i some.mobileprovision

Here, for instance, is the output of a provisioning profile for an app that uses SiriKit to trigger a workout:

napkin

As you can see, this is a convenient way to confirm the associations in the prov-pro, particularly entitlements, the app ID, and provisioned devices.

Mysterious crashes in your iOS 10 program? Check your info.plist

If you’re developing for iOS 10 and your app “silently” crashes (especially if it’s an older app), the culprit could well be the increased privacy requirements in iOS 10. Namepaces such as HomeKit now require specific privacy-related keys to be in your info.plist (for instance, NSHomeKitUsageDescription). If you don’t have them, the system automatically closes your application without an exception or Console.log message (if you run in the simulator, you may see a PRIVACY_VIOLATION notice in the stack trace).

Streaming a Web video to AppleTV with Xamarin

If you have the URL of a streaming video, it’s easy to display on an AppleTV, even though tvOS does not have a UIWebView (which would make it really easy). You have to use some AVFoundation code, such as:

[code lang=”csharp”]
var src = NSUrl.FromString("https://somevideo");
var asset = AVAsset.FromUrl(src);
var playerItem = new AVPlayerItem(asset);
var player = new AVPlayer (playerItem);
var playerLayer = AVPlayerLayer.FromPlayer (player);
//Might want to modify this so that it’s the same size as the source video
var frame = new CGRect (0, 0, this.View.Frame.Width, this.View.Frame.Height);
playerLayer.Frame = frame;
this.View.Layer.AddSublayer (playerLayer);
player.Play ();
[/code]

Note: This won’t work with normal YouTube page URLs since the YouTube stream URLs are not directly accessible.

The Half-Baked Neural Net APIs of iOS 10

iOS 10 contains 2 sets of APIs relating to Artificial Neural Nets and Deep Learning, aka The New New Thing. Unfortunately, both APIs are bizarrely incomplete: they allow you to specify the topology of the neural net, but have no facility for training.

I say this is “bizarre” for two reasons:

  • Topology and the results of training are inextricably linked; and
  • Topology is static

The training of a neural net is, ultimately, just setting the weighting factors for the elements in the network topology: for every connection in the network, you have some weighting factor. A network topology without weights is useless. A training process results in weights for that specific topology.

Topologies are static: neural nets do not modify their topologies at runtime. (Topologies are not generally modified even during training: instead, generally the experimenter uses their intuition to create a topology that they then train.) The topology of a neural net ought to be declarative and probably ought to be loaded from a configuration file, along with the weights that result from training.

When I first saw the iOS 10 APIs, I thought it was possible that Apple was going to reveal a high-level tool for defining and training ANNs: something like Quartz Composer, but for Neural Networks. Or, perhaps, some kind of iCloud-based service for doing the training. But instead, at the sessions at WWDC they said that the model was to develop and train your networks in something like Theanos and then use the APIs.

This is how it works:

  • Do all of your development using some set of tools not from Apple, but make sure that your results are restricted to the runtime capabilities of the Apple neural APIs.
  • When you’re done, you’ll have two things: a network graph and weights for each connection in that graph
  • In your code, use the Apple neural APIs to recreate the network graph.
  • As a resource (download or load from file) the weights.
  • Back in your code, stitch together the weights and the graph. One mistake and you’re toast. If you discover a new, more efficient, topology, you’ll have to change your binary.

This is my prediction: Anyone who uses these APIs is going to instantly write a higher-level API that combines the definition of the topology with the setting of the weights. I mean: Duh.

Now, to be fair, you could implement your own training algorithm on the device and modify the weights of a pre-existing neural network based on device-specific results. Which makes sense if you’re Apple and want to do as much of the Siri / Image recognition / Voice recognition heavy lifting on the device as possible but allow for a certain amount of runtime flexibility. That is, you do the vast majority of the training during development, download the very complex topology and weight resources, but allow the device to modify the weights by a few percent. But even in that case, either your topology stays static or you build it based on a declarative configuration file, which means that whichever route you choose, you’re still talking about a half-baked API.

Bizarre.