Node-RED is coming of age
I've been using Node-RED for over a decade now (here's a post to "prove it"), I've used it personally on Raspberry Pi devices, on robotic platforms, I've used it in Enterprise grade level clients across Europe & Africa to manage smart buildings, improve process flows & distribute / parse data to where it needs to be to make informed decisions. I even had it running on a prototype BMW i8 in Munich at one point, talking to an AI tool :-D
A couple of years back I even used Node-RED hooked up to a 1979 Atari 800XL to connect to the internet / LLMs, just for fun - but it did show just how powerful, quick & simple it was to get that all important "integration" working from an idea into a proof-of-concept to a demonstration.
I often hear the sentence, "oh, but node-RED is not scalable. It is not able to handle 50,000 messages a second processing, etc..etc..". Well, as I inform these ignorant people, "do we need to?". then I wait. I wait with the difficult silence and just stare at them. They then usually come back with some response, such as, Apache NIFI can do that. Now, I have NO problem with this other open-source offering. However, with everything, it all comes down to "what is it you are trying to do?"
Think of these as hand tools. This is a screwdriver, a philips screwdriver. It does this job very well. However, it is when people start to use it as a hammer or as a spade screwdriver, you should look to use a different tool, rather than sticking with "what we always use". Use the best tool for the job. simple.
Also, if you are processing 50,000 messages a second you might want to think about the design of what you are doing & how you are doing it (like I did in 2015-ish for Daimler/Mercedes with their self-driving trucks out in Germany, you use protobuff, Kafka & Zookeeper - and I STILL used node-RED in that equation too for processing & for the dashboard UI)
The other perspective is skillsets. Node-RED is very simple, easy and low-code JavaScript for a reason. The learning curve is short, it has a full open-source community backing, it has over 5000 active nodes that can pretty much do anything you want or need. If it doesn't exist, it is really quick, simple & easy to make a new one - or request for someone to contribute to make a new node, the community is really helpful & encouraging.
As to "it is not scalable, it is not on the Cloud, it is not secure" - welcome to FlowFuse. This is a company, setup & run by 1/2 of the co-creators of node-RED, they have made node-RED "Industrial".
They have answered all those doubts, all those questions, all of those restrictions & have cracked the adoption, deployment & usage in a scalable and industrial 24/7 usage of node-RED. Hmmm... that you, the client/customer can take over the management / modification / usage of the flows yourself, you don't need £2k a day Java coder "experts" who will just see you as a ca$h cow for years to come.
It's called common-sense & freedom. Would I put everything into production using node-RED? nope. Would I use it to accelerate turning an idea into a functional integration that proves that something can be done, in a short time frame (usually hours / days) - yep! that is the sweet spot.
Whilst you are still defining all the complex setup / definitions in NiFi, I've not only built, tested, deployed & proven the "thing" works, I've moved onto the next 10 challenges and are progressing through them at a rapid rate.
Check them out HERE: https://node-red-academy.learnworlds.com/
I encourage you to go checkout the 2 courses they currently have available for "free" on their website - I am sure there are many more to come. They give you a good spread of how best to use node-RED itself & when / where FlowFuse can bring in those extra elements - such as the DevOps pipeline automation, the RBAC controls, the ability to manage version control seemlessly. All those "things" that the good old Enterprise people want to have (but in honesty, most rarely every use properly!)
Also, as you've probably already (if not, why not?) got node-RED installed on your own machine, probably natively? ie. directly installed to your laptop / RPi.
I will assume & well, will encourage you to just always run this install / setup - it makes life SO MUCH SIMPLER!
Just run the single command (can be used for 1st install & for getting updates too)
Then enable it as a service & it's all done.
Now, if you want to use FlowFuse onPremise too, yep, you can do that, you can do a native install, but you can also use Docker too (I'm actually a fan of docker for personal use / testing / getting up & running). Follow THESE STEPS:
Start checking it out & working out where the boundaries are for what you can / need to do using node-RED & when you need to use FlowFuse.
----------------------------------------------------------------------------------------------------
Right, back to what I was ACTUALLY going to post at the start of this article - I wanted to test to see if I could ingest an image, use a simple TensorFlow (other models are available) to identify an item in the image and then get the metadata that contains what was detected, what the bounding box info is, determine if I could then send that off to another system & also show a preview of that annotated Image.
Yep - simples. It did take about 1 minute to process the 7MB JPG file that I was using. I deliberately used the photo of my van with a tree cut through the center to see what it would detect. I am sure with smaller images it would run faster.
REFs:
https://www.linux.com/news/using-tensorflow-js-and-node-red-with-image-recognition-applications/
Would I use this for everything? Nope, as the currently used model has only been trained on X amount of items. However, as this is node-RED, that node I'm using, I can review the source-code, I can clone it, I can make my own node, I can then use my OWN model, trained against my own items of interest & I can then make this fit for the specific purpose I want to use it for. Probably all within a single day.
As you can see, this goes beyond just processing & distributing data (Apache NiFi), I can now add intelligence / smartness into my flows.
UPDATE:
I installed FlowFuse natively on my UBuntu laptop (not via Docker as I hit a stumbling block with the DNS requirement that I wasn't going to faff with)
And very quickly was able to replicate my node-RED setup the user plugin that allows me to do some simple RBAC logic too.
Note - I've now got the benefit of using [SNAPSHOT] for version control. sweet.
TANGENT:
I was asked this week, "why are you not using Agentic AI instead?". sigh. you poor deluded individuals. You need to be removed from the hype of the Gen AI BS. Okay, I can say that with confidence as I was actually building, what is now termed as Agentic AI, back in Feb 2023 using LangChain (there might be a post about it somewhere? cannot be bothered to find a link as it really is not important), it became very apparent that the whole Agentic AI process is flawed. "Huh?", I hear you say. Yes, the infamous garbage-in, garbage-along & garbage-out is rife with this approach.
Here's a link to a recent tool - if you do insist on wasting your time with this endevour.
So, with Langchain you used to be able to set up a set of [tools] where you would have a "skill" for each area of interest, such as ToolA would be knowledgeable on InfoA, ToolB would connect to a DB and be knowledgeable on InfoB - and you could use a controller LLM to work out what you were asking, use the "tools" to determine what would be best to ask to get a response, take the response & pass it onto the next Tool in order to get a response, etc..etc... and indeed this is what I built..... and tested. extensively.
It is NOT rocket-science, it is about 30 lines of code - you do NOT need to pay companies gazillion dollars to implement this, do NOT believe the current hype / marketing BS.
And this was the enlightenment moment. It is not consistent. You would get variances as the underlying LLM model logic works on 'next word pattern matching', it is NOT INTELLIGENCE.
UPDATE:
Well, looksie here, there are researchers making it so that AI Agents actually create AI Agents -
https://www.emergence.ai/blog/towards-autonomous-agents-and-recursive-intelligence
That is so funny. Even as the "gold rush" was happening & Enterprise consulting firms are pushing the mantra that you need 1000s of "expert consultants" to help you to design & build these "Agents", the software is going to now just do that. This reminds me of the ever-lasting light bulb. "Industry" may very much obtain & bury it as too much of a threat to revenue. However, in todays social media society that might be unfeasible, so it just might happen. In which case, save the brain cells / neural pathways trying to figure out how to do it yourself, you've already been surpassed. As the old saying goes, selling shovels & hotel rooms is where the money is, not actually panning for the gold.
Oh, I should do a whole article on this subject itself, the greater you DIG the deeper down the rabbit hole you go:
There is a company that now removes the human Data Scientists. sure. why not. what they do is just a repeatable analytical pattern after all, isn't it? You can see this fitting in with the WEF 2030 strategy very well.
Anyway, back from that TANGENT and off to another one:
Now, if you just want a repeatable worklfow, there are many open-source engines out there that can help, for instance, ARGO or Apache AirFlow, these do a great job of being able to repeat a task consitently, they can call out to different systems based upon dynamic content, however, they are not considered part of the new "AI BS" that is being hyped as the saviour of human-kind.
Can you get these workflow engines to tap into the power of node-RED - sure thing! Now, that is where the value starts to comes in. That is where the smart comes in. That is where the, "I can trust this tool", comes in.
See, you need to still be hands-on enough to know what is BS & what is useful, where is the best place to spend your money - not just following the herd & wasting time, effort & money.
Happy Node-REDding!
Comments
Post a Comment