AI – Pause for thought

In the past week or two, views attributed to both Stephen Hawking and Elon Musk have been published, both in general questioning the value of major advances in artificial intelligence. Reading accounts of both, it seems to me that the issue is not so much artificial intelligence as artificial sentience.

You could not read this site without using artificial intelligence. It underpins pretty much any decision making software, and for this page to get to your computer, quite a lot of decisions get to be made network related software.

Decision making software decides what books Amazon recommends to you, what search results Google returns to you when you search for something like, for example, artificial intelligence. Ultimately the issue lies in our perception of artificial intelligence; it has moved on a long way from the ideas prevalent when Alan Turing wrote his seminal paper in the 1950s. At that point, I think it is fair to say, we had a vision of artificial humans rather than task driven machines. What we have been able to achieve in the last 60 years is huge in some respects, and relatively limited in many other respects. Ultimately, our successes have been because we have, over time, changed our understanding of what constitutes artificial intelligence.

There are a lot of intelligent systems floating around the world. Google’s driverless cars are amongst them, but even if you skip them, there are a bunch of driverless trams around the world (Lyon and CDG Airport in Paris as two examples). These systems make responsive decisions and not pro-active decisions. You may get up in the morning and decide to get a train to Cork; a train will never make that decision because a human makes the decision about where the train is going. It doesn’t, unless it manages to collude with any automated signalling system somewhere around Portarlington, decide to go to Galway instead.

I think it is fair to say that both Elon Musk and Stephen Hawking are far, far brighter than me so normally I would hesitate to suggest that they are perhaps worried about something which is less than likely to happen. Eric Schmidt has also spoken and he has pointed out that we are very far short of systems that match intelligence and he raised the one example that I tend to, although usually when I am talking about something else. Google have a rather infamous piece of research which I search for under the heading of Google cat paper. Google suggests this to me when I start typing it in which suggests I am not the only person to tag it that way. Anyway, from the Wired interview with Eric Schmidt

All that said, Schmidt also confessed that these machines are a lot more primitive than people would like to imagine. For evidence of that fact, he explained an experiment Google conducted a few years back, in which the company’s scientists developed a neural network and fed it 11,000 hours of YouTube videos to see what it could learn, without any training. “It discovered the concept of ‘cat,’” Schmidt said, assuming the tone of a disappointed dad. “I’m not quite sure what to say about that, except that that’s where we are.”

For me, the issue isn’t so much that the system limited itself to cat. If you know anything at all about YouTube, it’s almost a safe bet that after “human” the next thing any image recognition system is likely to pick up is “cat” purely by force of frequency. What matters is the sheer amount of work taken to get as far as learning “cat”. Schmidt does not mention this.

What the experiment actually did was obtain 10 million still images from YouTube videos and ran them through a nine layer neural network which ran on a cluster including 1000 machines (so 16,000 cores) for three days. According to their abstract, this is what they achieved:

Starting with these learned features, we trained our network to obtain 15.8% accuracy in recognizing 22,000 object categories
from ImageNet, a leap of 70% relative improvement over the previous state-of-the-art.

Now, ultimately, core piece of information here is that a) it was a huge effort distributed across 1000 machines b) took a long time and c) resulted in an ability to correctly identify 22,000 object categories from a big database in 15.8% of cases.

When Eric Schmidt suggests this is relatively primitive, he is right. Now compare it to how fast you can generalise the concept of a cat, or, for example, a new device (I like to use drones here). It is worth noting that Google specifically used unlabelled data here which is akin to a child wandering around a new world with no one telling him what anything is. Learning by experience if you like. Facebook is trying to do the same with labelled data however. They have a significantly greater success rate.

However, again it used large scale effort:

Thus we trained it on the largest facial dataset to-date, an identity labeled dataset of four million facial images belonging to more than 4,000 identities.

Microsoft, incidentally have some more research here. I haven’t read it in detail (yet) so am not going to comment on it. But they have made a lot of promises prior to publication, particularly with respect to accuracy in classifying animals, for example.

On the wider question of whether we should be worrying in much the same way as Elon Musk and Stephen Hawking are worrying; I am not sure the answer to that question is yes. Not at this point anyway.

One of my interests is in the area of natural language processing, and specifically language acquisition. We are miles short of generally usable systems here. As far as consumer systems are concerned, if you try to get Google to search for something which features a non-English name (ie, Phone, please play Alexandre Tharaud for me), you can forget about it. It does not work. My phone plays music by English rock bands but not French concert pianists if I try to use voice operation.

Yes, I can get my phone to list my appointments for the next few days simply by talking to it, but if you have spent any time using the Google Now at all, you will have run into some of the core issues with it: 1) your voice instructions are not locally processed and 2) your voice instructions are not guaranteed to be correctly processed either. Put simply, interaction is still limited. I still type search functions because it is faster.

Anyway, I wanted to pick up something Elon Musk said.

Musk is one of the high-profile investors, alongside Facebook’s Mark Zuckerberg and the actor Ashton Kutcher, in Vicarious, a company aiming to build a computer that can think like a person, with a neural network capable of replicating the part of the brain that controls vision, body movement and language.

It is worth noting, by the way that this is Vicarious. In certain respects, it is worth a look to see where they are at at present.

Put simply, there are a bunch of problems there and none of them are fully clear to being sorted. Arguably, we are reasonably close to building reasonable systems for handling motion related tasks – otherwise we would not be in the zone of having driverless cars. However, there is still a massive amount of work to be done in the area of vision, and above all else, language and then, a core issue will be getting systems handling these tasks to communicate effectively. We are a way from fully sentient artificial intelligences, for all the people who are working on them. Where we have had artificial intelligence successes, they have generally been task specific and those tasks have tended to be limited in some respect. We are a long way from building a truly sentient humanoid robot. When we build a self driving car, that is a monumental achievement in artificial intelligence. Said self driving car will never learn to come in and make tea. Artificial intelligence as it exists at present is pretty much scope limited.

However, there is a much wider debate to be had in terms of identifying what is practical and possible within a reasonable time frame, and what we want to achieve with artificial intelligence. This debate is not facilitated by Elon Musk suggesting that it is our greatest existential threat. It is entirely possible that in practical terms, it isn’t. Climate change is an issue. Resource misuse is another major problem. Inequality. Historically, every single major civilisation which has preceded ours has come one hell of a cropper at some point and left interesting looking landmarks behind for us to ponder and in no case was artificial intelligence the problem. It is entirely possible that before we ever get sentient robots we will run into problems. Even in itself, research into artificial intelligence has come a cropper in terms of obtaining funding.

But this shouldn’t preclude us considering what we want to enable and what we want to prevent. And in practical terms, how we ensure that bad things are never developed given that we have so far failed to prevent bad things being developed from useful technology. These are not, however, artificial intelligence questions. They are human questions.

The value of full time education

Brian Mulligan has a piece in today’s Irish Times, or at least, it has appeared online overnight anyway, questioning whether we can afford to send all our young people into full time education. It is worth noting that he is pushing online and distance options as a replacement for same, and that he is a lecturer and program manager in the Centre for Online Learning at the Institute of Technology in Sligo.

I wish to address a number of points he raises, but, in the interests of transparency, I will outline my own experience in this matter before proceeding: 1) I have studied full time in DCU for four years. 2) I have studied full time and worked part time at the University of Westminster for one year; 3) I have studied part time at DCU and worked full time for 2 years 4) I have worked full time and studied via distance learning with the Open University for 2 years and I studied full time at UCD, including 2 modules which were delivered online. I may not be a program manager for an organisation which is selling distance services but I have been a user of education services in most forms at this stage.

There is a trope in existence that when a story regarding some form of science opens with a question, the answer is usually No.

Brian Mulligan opens with a question.

Could it be that sending our children to college is an extravagance?

I think it is fair to say the answer to this question is actually No. No it is not an extravagance. In fact, if I had to say anything, the ongoing debate about the cost of education and what it brings is evidence, perhaps of knowing the price of things but not the value. Sending people to college is only an extravagance if you have an extremely narrow view of what education should be about.

The vision which Brian Mulligan sets out of young people going into menial jobs – as he calls them – and obtaining an education via some distance form enabling them to stay at home – is evidence of someone focusing on price rather than value.

As learners are mostly working and do not need to live away from home, they can more easily afford the fees, often with assistance from employers and with less subsidy from the State.

Quite.

There is no doubt that changes in technology may bring about changes in how education can be delivered, but this is no argument in favour of making full time education the privilege of the wealthy only, not if you want a healthy and reasonably equal society. One of the core values in enabling youngsters to leave home and go to college is that they learn to stand on their own two feet more quickly, and they meet a greater social mix; something which is a good thing, and highly important for an integrated society. It is no harm that people get to mix outside their own social circle; for society as a whole it is massively important. This is not a method of qualifying people as socially confident or not; what matters is ensuring that people are not constantly reinforced in narrow views in a homogenous conversation.

Should individuals and the State be spending or borrowing so much for what now could be seen as a pleasant rite of passage for privileged people? An extravagance?

Brian Mulligan is at liberty to consider his own education a pleasant rite of passage. He would make it a pleasant rite of passage for privileged people purely by removing from unprivileged people the right to and support for time to dedicate themselves to education. He wants to use modern technology to set society and education back 50-60 years again purely by allowing privilege to define your right to study full time. And he does this to support his organisation’s business case.

The trouble for me is this: I do think that it is not a rite of passage to go through some specialised education. I think it is highly suitable for some people, and less suitable for others. I honestly believe that we should have a decent and recognisably standardised apprentice system. I also believe that for young people who choose not to travel the academic route, we should have methods of enabling them to access what training they require. But I do also believe that money should not be the defining method by which we choose who gets a full time education in UCD and who does not. In a decent society, the marker should be ability and will.

That being said, I honestly believe that distance and on-demand education is something which is poorly provided and basically lopsided in this country. I mentioned above that I had done distance education via the Open University. I did this because at the time, there was no obvious way of studying for a mathematics degree part time from any university in this country. If you are looking for arts based degrees part time via distance, the provision is very poor. Most of what Brian Mulligan’s organisation offers is skills specific based rather falling under the broader term of education. Training rather than learning, if you like

I am not in favour of seeing distance and online education being sold as a replacement for letting our young people go to college. However, that is not to argue against it having a value for enabling all of society to have a continued access to education for the sake of education

Ireland has a serious need for a wider debate on education and training and the value we attribute to both. I have seen the opportunity cost to the country of economics dictating who gets to college and who does not. We also need to understand that sometimes, there is an argument in favour of education for education’s sake; for enabling people to access knowledge on an ongoing basis. Education does not end at the age of 21, 23 or 25 depending on how far you get through third levels into academia. Its value is not solely linked to your salary out of university because it also has a monumental impact on how you look at the world.

Learning Finnish

If you look at my CV or my About pages you will see that in 1998 I spent some time in Finland learning Finnish. It was four intense weeks of Finnish through the medium of Finnish, supported by the nice people in the Finnish Government who, at the time, funded two people to go to study at summer school in Rauma each year. I am not sure if they still do it and I don’t even know now how I found out about it. I was a temporary agent in Brussels at the time but I got the scholarship on the basis of being Irish.

Anyway, four weeks of intense study will take you so far, and I did some follow up work while I was still living in Belgium, but life took an interesting turn into computer programming and somehow, this habit of acquiring bits and pieces of human languages got set aside in favour of speaking directly to computers. I wrote assembler for a long time. And I learned Java, and I made projects happen with Javascript, Python and R. And SQL.

I did a postgrad in computer science last year which finished up around September and usually, when I do something like that, I follow it up with some completely different activity. Instead of it being some form of craft work (my last postgrad was followed up by a stained glass and mosaics design course), I decided to pick up Finnish again.

Finnish is an interesting language in a lot of respects. It is a branch of the Finno-Ugric languages which are generally thought to be unrelated to the Indo-European languages (although I believe there is some research in philology questioning this). Unlike most languages, it has an entirely logical spelling system with no irregularities. It has some grammatical oddities and structurally, it has some serious idiosyncrasies. Above all, it is a highly compacted language. I remember some of the very basic stuff, but I have more or less forgotten the verb and noun rules.

Being prudent, I have picked up the books I bought 15 years ago to study Finnish, mainly because I know they are good books, and also because getting decent dictionaries seems to be harder now than it was then. And this includes going through the Akateeminen Kirjakauppa book store online. The other thing I am doing, which is linked to some rather traumatic memories involving German, is reading a news story a day with the aid of a dictionary. This is massively challenging for several reasons, of which, having forgotten the verb and noun rules, identifying root forms of both can be impossible, and of course, it just takes a long time. I do, however, believe it is one of the more effective ways of broadening your vocabulary. It’s just not that easy.

In terms of language acquisition, some things are much easier now. I am not really going to talk about Duolingo (I have doubts about it as a learning tool for me anyway, and I don’t believe it offers Finnish), but the simple availability of media. Even for Finnish, there is a substantial amount of material available through Youtube videos, for example. There are a number of radio stations available via TuneIn. YLE, the state broadcasting service does a special Easy Finnish news report which is what I use as raw material for my reading exercises. At the weekend, I think I will be able to watch skijumping and ice skating in Finnish.

When I was learning French, to get any media at all, I used to hide in a car which had a long wave receiver. At this point, it doesn’t massively matter how much I understand, only that the amount I am understanding is growing on an ongoing basis. My passive vocabulary will grow much more quickly than my active vocabulary and this is not all that surprising since this morning’s news story was about unemployment and part time work.

The interesting thing, from my point of view, is how much hard work goes into language acquisition. Being absolutely honest, it is harder work than learning programming languages.

And yet, in certain respects, it is very rewarding. One of the interesting things about Finnish is how sentences are structured and how that might suggest a completely different way of looking at the world. I find it fascinating purely from that point of view, never mind being able to converse with people in Stockmann when I go shopping there. In a lot of ways, I am really sorry I set it aside for so long. I am having fun with this.

learning: encrypting text

A while back, one of my friends on twitter introduced me to NaNoGenMo and late at night, I started thinking about what could be done with such a project. As they cited the possibility of 50,000 meows as an example of a possible successful project, I decided that a novel consisting of 50,000 different words making absolutely no sense whatsoever was a possibility, and decided to find a base text of 1000 words, and build an encryption algorithm that would generate 50 different encrypted versions of the text, and that would be it. And I would write the encrypted text generator in Python.

The primary reason I do stuff like this is for learning reasons and very often, you wind up learning more about how you look at a problem, rather than about whatever programming language you use for projects like this. More often than not, you’ll find a little bit of functionality that you didn’t know existed. And if you are really lucky, looking at stuff like this opens doors for you to look at things in more detail.

I am not an encryption specialist (yet) so effectively, I wanted to find some way of turning cleartext into something obviously encrypted but without it being too easy to immediately decode. The angle of attack I specifically wanted to block was frequency analysis. (It’s 10 years at least since I looked at encryption techniques but I remember some of it).

So I looked at building an algorithm which amounted to reducing the number of output letters, but in a random manner. Each individual run of the encryption algorithm generated a random number which was the total number of letters which could be used in encrypting the clear text, and for each letter generated random number, which represented which letter the cleartext letter would map to. I also generated a key of the mapping of original letter to encrypted letter, performed some minimal hiding of reality, and ran the algorithm against some text. It worked beautifully and what’s more, it was very obvious that you couldn’t see what had happened to the text.

Where I ran into a problem was in decrypting each piece of cipher text. When you basically reduce the dimension of available letters, regenerating the mapping from a smaller group of letters becomes a difficult to fix problem, particularly if you only have one example of the encryption algorithm. I could not actually produce a piece of code that immediately decrypted any individual piece of cipher text. The key generated by the encryption algorithm was a one way only key. So I have been pondering this problem in the meantime and I’ve concluded that the algorithm may yet be breakable if you have several examples of the same text encrypted, plus the matching keys and some willingness to go messing calculating the different encryption dimensions. In short, while it’s relatively straightforward to encrypt the text, you need many examples of the algorithm generating different keys plus the associated encrypted texts to break back in. I have not yet implemented this but I will look at it as a later problem.

In the meantime, key learning outcomes from this exercise:

  • encryption algorithms are easier to design than encryption with matching decription
  • Python has a useful string translate function which I was happy to find. I have used something similar in assembler programming for changing encoding between different character sets.
  • after all this, when I started reading up on cipher algorithms again, I discovered that there exists a pycipher library which implements a bunch of standard cipher algorithms. Even so, the existence of this does not mean I won’t, at some stage, have a look at implementing one of the Enigma or, possibly, one of the Lorenz ciphers just for the hell of it.
  • I want to read up on cryptography again. It’s been too long.

The github page for the project is here.

 

 

Recommendations at Etsy

Robert Hall, one of the data engineers at Etsy, a large online craft market place, has written a comprehensive overview of how they manage recommendations. It’s a very interesting piece in that it’s quite open particularly as relates to something which could be considered commercially sensitive, and it broaches on both the mathematics and infrastructure side of things.

Read it here. The Etsy Code as Craft blog is well worth a read in ongoing terms.

this is about data and technology and where I interact with both