Canadian Government Executive - Volume 25 - Issue 02

: : task : : : : data : : : : agent : : : : parameters : : : : learning methods : : : : model : : Typically, al�orithms tell a com- puter what to do for each situa- tion it faces. Predetermined pat instructions do not �uide machine-learnin� al�orithms. Instead, a statistical model identifies patterns and relation- ships in data, which then inform decisions. Insofar as assumptions are embedded in the model, the a�ent is said to lack “autonomy.” Patterns and relationships are inferred from observations (data). A smaller subset of trainin� data may be needed initially. Lar�e quantities of data (“Bi� Data”) are required ultimately. An a�ent is anythin� that can interpret data and act on it, such as a robot with physical sensors (camera) and actuators (arms). Or a “smart” household appliance. Or just software, as with tailored recom- mendations from an online store or an app that identifies the content of photos to or�anize them. What problem is the a�ent to solve? How is that problem defined? What parts of the environment are relevant? Answers to these questions define the task to be performed. For example, if the task is drivin� a car, the a�ent must �rasp the destination (�oal) and the rules of the road (constraints), plus be able to control the vehicle and reco�nize objects along the way. Machine learnin reco�nizes patterns and anom- alies in data in order to make better decisions. For example, it can make predictions based on patterns in historical data. It can �roup people or identify individuals based on patterns of similar- raises privacy concerns. Then there are worries about bias . There have been hi�h-profile cases of machine learnin� discriminatin� unfairly, makin� offensive comparisons, and blithely i�norin� important distinctions. So what is �oin� on? ity and difference. It can even spot suspicious behaviour and sound the alarm. Despite the bene- fits, machine learnin� is tri��erin� alarm bells of its own. The lo�ic behind decisions is not obvious, raisin� accountability concerns. Reliance on data The way an a�ent learns can be �rouped into four basic methods. The list of biases builds on Harini Suresh & John V. Gutta�, “A Framework for Under- standin� Unintended Consequences of Machine Learnin�,” ArXiv : 1901.10002, 2019. SUPERVISED LEARNING The a�ent learns to associate cate�ory labels with various examples that are labelled (trainin� data). The patterns found in those examples help the a�ent infer what cate�ory a new (non-labelled) object falls into. That inference can then be used to make decisions and complete a task. Some mistakes will happen, especially if trainin� data was not available for an object. UNSUPERVISED LEARNING There is no training data. The model measures similarities and differences of objects based on various features. Objects are then clustered into �roups. Many photo�raphic applications rely on this type of sortin�, for example. SEMI-SUPERVISED LEARNING A combination of labelled and unlabelled data may be used. A promisin� approach is “cooperative” learnin� that asks for help when it is not confident and adds labels to unlabelled data when it is hi�hly confident. REINFORCEMENT LEARNING In dynamic environments, the a�ent can learn with exploratory trial and error. Initial forays are more or less random. A reward function is then used to apply rewards and penalties as the a�ent works towards a �oal. If the task is complex but intervals to reward pro�ress are long, more human �uidance (“reward shapin�”) is involved. That undermines the ability of the a�ent to apply learnin� to new environments. The model will perform statistical procedures on data. These procedures are variations of ones that statisticians are familiar with, such as re�ression- and cluster analyses. These procedures have settin�s (“hyper-parameters”) that have to be tested and tuned to optimize task performance. Labelled validation data can be used to check the model and tune it. Humans often check complex models and label ambi�uous data too. 16 / Canadian Government Executive // April/May 2019