Blog powered by Typepad

« "Shurely shome mishtake here!" | Main | Your New Year's Eve Funnies - no, really . . . »

Sunday, 29 December 2019


Feed You can follow this conversation by subscribing to the comment feed for this post.

Here's one that's going to run and run ...

I'm right in the middle of my first live data, i.e. non-lab, AI projects just now. My hope was that it would be another mountain of complexity I could bury myself in and charge extortionate rates for the next decade, y'know, as you do in the "land of the blind, where the one eyed man is king" of IT. How wrong I was (making a bit of a habit of this, aren't we? ... Ed).

Imagine you've got a table of suppliers with names, addresses and bank accounts, 8,000 of them. However, there are many duplicates where busy purchasing teams have keyed the same supplier in twice or more times, with slight differences like Rd instead of Road, the first address line in the second address line field, some with postcode some without. And sometimes the same supplier trading with a different name but the bank account being the same identifying that they are in fact the same supplier. Big manual cleaning exercise for the admin staff combined with big programming exercise for the programming staff - err, that's me and my mates btw! Lovely jubbly, dollars all round, and jobs for life. Well several months work at least.

So, in an idle moment I thought I'd see what this 'ere Microsoft AI package called Azure Machine Learning can do.

I started at 9am, dropped 3 widgety things onto a page, left the 3 widgets with default settings, and dragged 3 arrows between the widgets to connect them together. One widget was my data - the 8k suppliers table I'd uploaded. The second widget represented the AI algorithm that was going to be used for the grouping of the suppliers into duplicates - nothing tricky, just a "blackbox" with a fancy name: "K-Means clustering". You didn't need to know anything about it. And the third widget was the processing and output - the "Go do, that voodoo, that you do, so well" and give me a download to collect the result, widget.

At 12 noon on the same day I inspected the result. By George and all the Saints: a nice neat little column had been added to the right of the existing columns in my table of 8000 suppliers, and therein was a number for each duplicate group of the same suppliers.

That's my programmer mates jobs up the Swanee!

But here's the punch line: I showed how I did it to the suppliers data manager who needed the job done. Took about 15 minutes, at the end of which he said, "That's easier than Excel, I could have done it myself - you're fired Duff!"

Bloody 'ell, eh? BoJo and Cummings need to hurry the fk up with that Universal Basic Income thing, coz it ain't just Northern shop floor proles gonna be jobless tomorrow ...


This "woke" movement feeds on its own short comings and failures. Wokeness can change daily and "cancel" anyone and anything at any time.

Ref that DT article, here's the rub ...

When you include the "Race" column in the table of data fed to the AI algorithm "blackbox" along with all the other columns like Age, Home Postcode, Number of Previous Offences, etc., in exactly the same way as I did with my supplier duplicates job, you get say a 96% correct prediction of re-offending, bail-jumping, or whatever.

If you remove the "Race" column, the correction prediction rate drops to say 65%.

So this serves the choice up to society (whatever that is, I'm with Maggie on that one) with all the obfuscation removed: Truth vs Positive descrimination - which is more important to "society" then, eh?

Using an AI algorithm and including "Race" more accurately predicts re-offending and bail-jumping. Period.

Which is it to be then: Safe or Woke?

Right in your face now ain't it?


I'm guessing the end result of woke and wokeness will be a heavy burden on the mental health industry. Truth or Safe will be just another way of saying reality, and culture war comes down to a fight to see who will define reality.

Roger that, Whiters.


Top 10 influential books of all times list is interesting. Seems "The Pilgrim's Progress" could have had an honorable mention though.

Thank you, JK, for that excellent link to the wise words of Rod Liddle but you have been a.w.o.l. for a few days - and don't tell me it was because of Xmas when we all know you have been up in them thar hills with Barney Magroo, 'Purveyor of Fine Wines and Spirits to the Gentry'!

Been in the doldrums David.

As in the nautical sense you understand - "calms, squalls, and light, baffling winds" - double on the baffling winds.

Still though I remain a regular.

Some music for you David, hit me where I live just now. Enjoy

Sorry, JK, but that miserable dirge "Cryin'" nearly reduced me to tears!

On the eve of a New Year we are still "on fire" in multiple locations. 44C today, 23C tomorrow. The "gerbil worming" brigade are going nuts blaming the fires on "climate change" and were sort of bewildered when it was pointed out that a lot of them have been lit by arsonists and that we have had worse fires and weather extremes.

This wide brown land has always been subject to heat, fire and flood.

A Happy 2020 to all of you.

Great to hear from you, AussieD. I have just trudged up the stairs after eating my breakfast and watching on TV the cauldron that seems to have enveloped parts of your country and I was about to pester you with enquiries as to your well-being. Don't so much 'Hang tough, mate' as 'Hang wet'! Please keep us posted as and when you can.

AussieD, stay safe and keep in touch when you can.

It would be a good idea to give up on criticizing "woke". I read several lefty blogs and no one has ever used the term. In the real world - the world outside the right wing media bubble - it's only used sarcastically:


The act of being very pretentious about how much you care about a social issue"


You're reading, apparently, the wrong sorts of blogs.

Or perhaps, you're bereft of black acquaintances.

Here Bob

That'll woke your ass up. Or, 'See'n it real' as the Cuz'ns say.'

(Note to my Brit pals - Tain't proper see'n as I white I calls the Bro's as they say of each other so we here'n Arkansas at least knows Cuz'n is bein' the exack.

Knowin' as I do sometimes David calls any and ever'body c'rost the pond Cousins.)

I'm sure you have noticed that David says "Cousins" with affection?

I have Whitewall, why I ensured I made the effort at phonetically spelling.

On another topic, you been able to connect to Dip's site today? I've tried two browsers but I'm continuing to get a pop-up warning of Trojan.

No problem at all even tonight. Just tried it. Maybe you're not holding your mouth right?

David, Aussie D, and all, I should have thanked you for your kindness the other day but I was mindful of the fact that I'll never be a regular commenter here because I rarely agree with anything you say, I'm a woman, and I can't be bothered to argue. We've already made our minds up. About everything.

There's something about the immediacy of the current crisis here in Australia. It does focus the mind when your house is about to burn down and you need to decide whether to stay or go. If it was just about the real and present danger we could probably just gat over it after the immediate threat has passed..but this year is different. The bush fire season started early, the fires are huge and, without rain, unstoppable. The thing is that nobody can breathe. I have a propensity to asthma but my husband has not. Last night, half awake I could hear a strange noise. It was him, wheezing loudly. He doesn't snore.

I'm a believer in climate change. I totally think that something has to be done. I also acknowledge that the Australian economy could be destroyed by doing too much, too soon. Which is not a reason for doing nothing.

The thing is, if you don't admit that there's a problem caused by what humankind is doing then you don't have to do anything about it. So just carry on. At some point, it doesn't really matter and we'll all be forced to face reality even if it means that our investments take a hit. You can give in gracefully or wait for it to happen in due course. Either way, get ready.

BTW, Aussie D , there have always been crazy arsonists. Most were daft boys having dare and it was easily extinguished. More disturbing are grown-up lunatics who want to fight a fire, so they light it. What's different now is how these fires spread because everything is so dry. We live on an acreage and have a huge capacity for water storage. Five minutes ago our man left us..we've just taken delivery of thousands of litres of water because we've had no proper rain in months. We're coastal, wet, and green Well, we were.

It's changing.

Nae worries Mary, you really might as well stick around, you do see us treating "our Bob" nice and I'd dare guess he's far more disagreeable than you could ever be. (Well, some of us do better than others of us at treating Bob nice I'd allow.)

Are you aware of our (I'm in the US Mary) California? Wildfires are a problem out there too - I'm in the interior of the US and California is about 2000 miles west of me. Anyways California's got a slew of climate change believers too. Matter of fact I myself consider there's something to climate change it's just that my thinking takes into consideration timescales (writ large - even unto the geological).

Trouble is sometimes individuals limit their thinking to that of a single human life-span and that's damned short. However I'm limiting my critique for my purposes here to the USA's California and my point is "most Californians of today" can't conceive that when that state's pioneers first stumbled into the Imperial Valley it wasn't "lushness" that greeted them rather it was desert. Bleak and very arid desert at that. And though hard to imagine today given that, today's Imperial Valley of California might justly be described if not the, at least one of The Breadbaskets of the World had not some very intensive infrastructure projects not been put in place none of that would have been possible.

But now California's infrastructure is crumbling and what worked so impressively through the past forty or so years isn't any longer. Worse for prospects is there's damn few around California anymore skilled to the point of being able to restore its former but fleeting glory - they've taken up marijuana horticulture and basketweaving and liberal politics and having their water delivered.

In other words they've forgot what nature's state is in its very essence and that is flux. Science is too, generally speaking, always in flux too - why when I was a kid the science insisted I'd be a mile or two under a glacier and California would be fallen off our continent.

I'm still waiting for that. California's falling off but not so much me being up to my mountaintops in ice I mean.

Mary, the fact that you do NOT agree with much of what is written here makes you a valued commenter. Likewise, (he writes through gritted teeth!) with tendentious Bob!

On another matter, you write, "I'm a woman, and I can't be bothered to argue." Never argue? Really? Will you marry me? I'm not much of a bargain but at least we have non-stop rain 'down 'ere in Zunny Zummerzet'!

I'm a believer in climate change. I totally think that something has to be done.

I believe in climate change too. Climate change is a good thing. It's been around for 4 billion years and without it life wouldn't have developed and been sustained.

In so far as climate change is harmful to mankind enough has already been done by capitalism to make any further effort a low priority ...

Some takeaways: -

1. A reduction in climate and weather related deaths from circa 500,000 a year to 20,000 in 100 years.

2. The individual risk of dying from climate-related disasters has declined by 98.9%.

3. More people die or are seriously injured on the roads in Britain as die worldwide from climate and weather related events.

The much higher priority is stopping and bringing to justice the perpetrators of the crime against humanity of climate change alarmism. They and it are already killing 600,000 Africans a year by denying central Africa a fossil-fuel powered electricity grid and thereby obliging people to cook on open wood stoves and die from lung disease ...


Look at this distorted piece of garbage from the Graun ...

Africa at last giving the middle finger to the climate alarmist murderers and going for fossils. And what half-baked analysis does the Graun spew out?: 48,000 excess deaths to be caused by the pollution of the new power stations.

No mention of the 600,000 lives that will be saved by getting electricity to everyone's homes for a net benefit of lives saved of 552,000.

Compared to captialism's 500k to 20k deaths per annum reduction over 100 years it'd be like taking Africa from 1919 to 2010 in a decade. That'd be something amazing to celebrate.

And Oz could sell them the coal and might be able to afford some more fire pits / trenches or whatever to reduce the climate and weather death toll by one or two more.

What is it about development and progressive processes that is such a blind spot and / or anathema to leftist authoritarians? Extraordinary, and as for the irony thermometer the readings off the effing scale.


SoD, I'm worried about you! Just lately, now the Brexit problem is solved, you have been writing some intelligent comments. Keep it up, my boy, and I might put you back in my will!

The comments to this entry are closed.