Apple's Vision & Machine Learning - Simplified

 

I am adding vision to my Sidekick bots. But for a start, I'd like to evaluate the technology first. 

The Goal

The goal in this article is to simplify the building block for using Apple’s Machine Learning framework and its Vision framework to a bear beer bare minimum - get it to work and easy to understand. Then, from thereon we can expand our understanding to tackle more complex work.

The plan

1. Use Xcode Developer Tool to train ML Model to classify hand-posture images
2. Get images from device’s camera
3. Use our trained ML model to classify the images captured from the camera
4. Re-train our ML model, i.e. make it more accurate
5. Bob is your uncle and the world is our oyster. Let’s do something cool about this!

More: https://primecoder.github.io/VisionML-Simplified/

Migrating to Swift 6

Enabling complete concurrency checking in a module can yield many data-race safety issues reported by the compiler. Hundreds, possibly even thousands of warnings are not uncommon. When faced with a such a large number of problems, especially if you are just beginning to learn about Swift’s data isolation model, this can feel insurmountable. 

(from: Swift.org's Migration to Swift 6)

 

Apple Intelligence - Getting Start

WWDC 2024 launched over a week ago and my hands had been itchy wanting to dig into it. Hey not everyone is lucky to be sent in - to spend a week at the conference! 😔 So, for me, Youtube is my friend. I have just had a chance to get my hands on to the development this week.  This year Apple has released quite feature-packed roadmap for their hardware and software. But the 2 main features I got excited the most are:

• Apple Intelligence, and
• Swift 6

First thing I had to check was that my old trusting Mac is still good. See System Requirements section below. Next, I needed to clear some HDD spaces for development. The rest turned out quite easy and quick - all you need is Xcode 16 Beta!

Let's get started! 

Development Update: Sidekick Build 50 - Rubber-duck Bots

 

 

"Rubber Duck Debugging"

A term coined from software development processes where you describe your problem to an inanimate object, say a rubber duck, then Bam! A faint-halo forms over your head. You hit a self enlightenment moment! Your problems become clear as crystal, you understand it, and now you KNOW how to solve it! Problems are gone. You live happily ever after. End of story.

What is Neural Network? - Can Shrek get his swamp back?

  

"Layers!".

"Onion has layers. Neural Net has layers." 

That what Shrek said on his quest to get back his swamp. (or did he?!)

What is Large Language Model (LLM)?

 

Imagine you were trying to teach a child (robot) to learn how to speak. You might start giving it a very simple sentence with just a few words. For example, 

"He who asks a question is a fool for five minutes; he who does not ask a question is a fool forever."

Bot Conversation Mode - Keep it simple and slay!

"Hey Jarvis. You kind-da boring."

🙀 OMG! How could I even say this to my bot?!

Yes, I did. And the truth can be hurtful. Sidekick app's UI (up to build 43) is kind-da complicated - so, I admitted.

"Can you make an app be more interesting by making it doing less?"

Bot's Compound Analysis - "Jarvis, get my gun ready."

"Imagine that you can quickly create a machine-learning bot at ease for your needs. Imagine that if you can then combine and reuse them for each of your specific requirements. Imagine that if you can share them. Imagine the excitement seeing them grow, continue learning and serving communities at large. Imagine seeing them being liked - while you sitting back and seeing them working for you. Perhaps, you can reap  some rewards along the way. Imagine..."

 

Introducing Classifier Bots

I have added more Machine Learning bots. They are of labelled learning types. 

Bootcamp Development Update (build 11)

 

Oh mine! There was lots of work done in the background to support the features below.