Technical Implementation
Introduction to Technical Implementation
Let’s talk about the technical aspects and implementations of AI employees. Many people ask me, “David, there are so many platforms where we can create AI agents – which one should we choose?”
Newo.ai Platform Differentiators
Our platform isn’t creating its own LLMs – that’s not our focus. We provide you with an out-of-the-box, simple, and nicely designed experience for connecting your AI agent to the physical world.
We give you four major differentiators: OmniChannel, OmniFlow, and OmniUser with physical presence – all aspects that convert your AI agent into an AI employee class of agent. These features are accessible through our new builder right out of the box.
Obviously, our platform, Newo.ai, utilizes all the best-in-class LLMs: OpenAI, Anthropic, Llama Grog, and Gemini – all top-tier models are available. So, if you’re a professional engineer working on AI automation, these are the tools you’ll use.
Real-World Implementation Challenges
Let me share a real case study. If you tried to create an AI host for a restaurant using a general-purpose AI agent platform, could you do it? Yes, you could. But how long would it take?
When we started this journey almost five years ago, even last summer (2023), our internal estimate was that based on any reasonable agentic platform, we could create an AI agent for a restaurant in a month. Some engineers said, “Give me two weeks” or “Give me a week.” More experienced engineers were confident they could do it faster. But here’s the reality: you might create a prototype quickly, but testing and refinement would take at least another three weeks.
Do you know how long it took us to make a robust, production-level AI host? A year, with ten people, and even then, our first simple restaurant implementation wasn’t perfect.
You might ask, “Why did it take so long? You have 30 years of experience in AI, a PhD, and you’ve been working with your team for 10 years!”
By the way, check out “The Most Extraordinary House in Silicon Valley, Dr David Yang” on YouTube to see our robotics work with RoboDogs, Abigail and Leo, and our six moving walls with human-like, emotional AI morphias.
The Devil in the Details
Let me break down why it’s so complex:
- Booking System Intricacies
- Simple booking might take a week to implement
- But people speak differently: some slowly, some fast
- Background noise and children’s interference
- Changing minds mid-booking
- Various booking system requirements (phone, email, two-factor authentication)
- Large Group Reservations
- Lead time considerations
- Different protocols for immediate vs future bookings
- Special manager approval requirements
- Menu preferences and equipment needs
- Corporate event specifications
- Additional Complexities
- Private dining arrangements
- Catering services and service areas
- Lost items handling
- Dynamic parking situations
- Emergency complaints
- Vendor communications
- Spam filtering
Our initial estimate was off by a factor of 100. We thought one or two people could create an AI host using advanced technologies like Langchain, AutoGPT, or OpenAI’s Assistant API in a month. We failed miserably. Fortunately, our first restaurant owners were my friends, so they just said, “Come back in six months when your AI host is ready.”
Technical Challenges: Date and Time Calculations
One surprising challenge was the date and time calculations. Our AI agents, as sophisticated as they are, sometimes struggle with simple date calculations. They might get confused about “next Sunday” or “in half an hour.” While they’re correct 90% of the time, that’s not good enough for production use. We consider “stable” to mean one mistake out of 100, but initially, we saw errors in 5-10% of cases.
Platform Selection and Implementation Strategy
If you’ve decided to create an AI employee class of agent, you have options:
- Use general-purpose agentic platforms: takes 2-6 months to reach production-ready state
- Consider platforms with built-in phone and verbal communication capabilities: takes 4-8 weeks to reach production-ready state
- Use specialized AI employee builders like Newo.ai: takes 2-8 hours to reach production-ready state
Remember, connecting a phone isn’t enough. You need integration with:
- Calendars
- Emails
- Instagram
- WhatsApp
- Slack
- Microsoft Teams
Recently, Newo.ai announced a creator component that builds agents automatically. You just provide the URL of your target business or find it on Google Maps, and our system creates a basic experience in three minutes. This combines with our builder platform for ongoing improvements and customization.
Finally, Newo.ai platform includes proprietary technologies like magic browser and magic API integrations, allowing AI agents to work with legacy systems through browser window understanding and keyboard/mouse action imitation. This enables seamless integration with existing booking systems and CRMs, even without technical APIs.
Platform Selection Considerations
When choosing a platform for AI employee development, you face a strategic decision. You can use general-purpose AI agent platforms, but remember what I mentioned earlier – it could take months to reach a production-ready state.
Let’s talk about deployment costs. Ideally, you don’t want to charge for deployment, or at least you want to limit these charges. Why? Because otherwise, your sales cycle will go on forever. You want to deliver something free of charge, quickly – those low-hanging fruits that create an immediate “wow” effect. You don’t want to tell your potential customer, “Just give me $20,000 for professional services, and I’ll show you something in two months.”
Integration requirements are another crucial consideration. You need to connect:
- Phone systems
- Calendars
- Emails
- Social media (particularly Instagram, where many leads come from)
- WhatsApp
- Slack and Microsoft Teams
- And potentially many other communication channels
Newo.ai – Human-like AI Employee Builder
Newo.ai Three Architectural Layers: 1-Click, No-Code, Low-Code
The Newo.ai platform enables users of any technical level to create and modify human-like AI agents through three distinct layers:
1-Click (A-Level)
This is designed for business owners and non-technical partners. You can customize AI employees built on Newo.ai simply through parameter settings. Nearly 90% of all necessary changes can be made without even opening Newo Builder. This includes:
- Product and service settings
- Pricing adjustments
- Working hours
- Email addresses and phones for escalation
- Message routing
For example, you can easily set up complex rules like: “During business hours, emergency calls should be forwarded to phone XXX, while after hours, send an SMS to number YYY, and in either case, send an email summary with the transcript to address ZZZ” – all without touching the builder.
No-Code (B-Level)
At this level, you provide instructions in natural language. People without technical education (including high school students and humanities majors) can master this level in just a few days. You can modify:
- Specific conversation scenarios
- Order calculation methods
- Lead form field requirements
- Slot availability checking timing
- SMS sending triggers
This is where you define your AI employee’s job description through workflow and workflow steps for all your business processes.
Low-Code (C-Level)
This is the deepest level of the system, where you can:
- Implement any creative AI automation ideas
- Create complex asynchronous integrations with the external world
- Mix data from various parts of reasoning and memory of one or multiple personas (both human and agent)
A programmer with 2-3 years of experience can master this level in 1-2 weeks. It provides ultimate flexibility while maintaining the ease of use that characterizes our platform.
Each layer builds upon the previous one, creating a comprehensive system that can handle everything from simple customizations to complex integrations – all while maintaining the human-like interaction that defines our AI employees.
Advanced Platform Features and Architecture
Let me walk you through some really cool features we’ve packed into Newo.ai. Trust me, this is where it gets exciting!
Integration and Communication Capabilities
You know how everyone talks about integration these days? Well, we’ve gone all in. We’ve got everything hooked up:
- Phone lines? Check.
- SMS systems? Absolutely.
- Webhooks? You bet. And here’s the cherry on top – we’ve got this amazing thing called NeoRAI (Retrieval Augmented Instructions). Yeah, it’s patented, and yeah, it’s as cool as it sounds!
Sophisticated Analysis and Logging
Here’s something that blows my mind every time I think about it – we analyze EVERYTHING. I mean everything:
- Every single turn in a conversation
- Every response from both agent and user
- Every little interaction
Why do we go to all this trouble? Because it helps us make the system:
- Lightning fast (bye-bye, latency!)
- Super accurate (no more random hallucinations)
- Rock-solid stable
- Predictable as your morning coffee
The Persona-Actor Architecture
Now, this next part? This is my baby. It’s what makes our agents true “AI employees” instead of just chatbots with a fancy name. We call it the persona-actor separation, and let me tell you how it works.
Think about your friend Michael (or maybe you’re Michael – hi, Michael!). In our system, Michael is a persona – a real person. But here’s where it gets interesting. Michael might talk to us through:
- His home phone (maybe late at night)
- Work phone (during those long office hours)
- Mobile (probably while grabbing coffee)
- SMS (when he’s in a meeting)
- Email (for those detailed conversations)
- Live chat (when he’s browsing the website)
The cool part? Each of these “actors” keeps its own memory of conversations, but – and this is the magic part – our platform can mix and match these memories when needed. So if Michael calls us, switches to SMS, and shoots us an email, we remember everything! It’s like having a personal assistant with an incredible memory.
Advanced Speech Processing
Let’s talk about something we all struggle with – you know that awkward moment when you start talking, and the other person isn’t actually finished? Even humans mess this up all the time!
We’ve cracked this nut with some pretty clever tech. Our system is constantly watching for those natural pause points, but here’s the clever bit – we don’t wait around twiddling our thumbs. When we think someone’s done talking, we start cooking up a response. But if they keep going? No problem! We can stop mid-thought and start over, just like a human would.
Business Portal vs Builder
We’ve got two main ways to interact with our system, and trust me, we’ve thought this through carefully.
There’s the Builder – that’s where all the techie stuff happens. But here’s the thing – if you’re a business user, you probably never need to see it. It’s like the engine room of a cruise ship – it’s important, but passengers don’t need to visit!
Instead, we’ve got this beautiful Business Portal. This is where the magic happens for everyone else. Whether you’re the boss, a manager, or someone who’s just “hired” our AI employee, you get this gorgeous dashboard with:
- All your conversations are laid out nice and clear
- Quick summaries (because who has time to read everything?)
- Reports showing how much extra money you’re making
- These really sweet charts show all sorts of useful stuff like:
- How many new customers you’re getting
- Who’s coming back for more
- For restaurants, all your bookings and cancellations
- All that good stuff!
And get this – we’ve even hooked it up to Google Sheets! Everything lands there automatically, so you can:
- Add your own notes
- Copy stuff around
- Link things together
- Set up your own automation if you’re feeling fancy
Outbound Communications
Here’s something really cool – our digital receptionists don’t just sit around waiting for the phone to ring. They can reach out too!
It’s super simple:
- Just give them a spreadsheet with the basics:
- Who to call
- When to call them
- What to ask
- Hit the magic “start” button
- Then sit back and watch them go!
They’ll make calls, send messages, chat with people like a pro, gather all the necessary info, and even invite folks to your events. And if someone doesn’t pick up? No worries – they’ll try again later, as many times as you want them to.
But here’s my favorite part – the omnichannel magic. Your AI employee can:
- Start with a friendly SMS
- Follow up with a call if needed
- Wrap it all up with a nice email
It’s like having the world’s most persistent (but never annoying) assistant who never gets tired and always knows exactly what to do next!
Pretty cool, right? This is what happens when you really think through how people actually communicate in the real world!
Newo.ai Creator Solution: 3-Minute Agent Creation
This is where I’m particularly proud of what we’ve achieved — we announced our creator component – a specific part of our platform that creates agents automatically. How does it work? You simply provide the URL of your target business or find the business on Google Maps.
Our creator – a set of agents built on our platform – learns and processes all available information from the internet. It then creates your basic experience right away in just three minutes. When we first accomplished this, we could hardly believe how elegant the solution was!
The beauty of our approach lies in its dual nature:
- The Creator part builds a sophisticated, production-level agent in three minutes
- The Builder platform allows for continuous improvement and complex modifications
Newo.ai Magic Browser Integration: Accessing Work Data Without API
One crucial element I want to emphasize is the ability to interact with browsers. When creating an AI employee, communication channels like phone, SMS, email, and calendars are important – but you also need to empower your AI agent to work with existing software, including legacy CRM and booking systems that might not have technical APIs.
Our proprietary technologies – magic browser and magic API integrations – allow the AI to:
- Understand browser window content
- Simulate keyboard and mouse actions
- Interact with legacy systems
- Create bookings in existing systems without direct API access
Additional Features
You’ll get access to a comprehensive platform where you can:
- Listen to all voice recordings
- Review transcripts and call summaries
- Monitor the category and type of calls
- Additional Secured Revenue (ASR) reports
AI Employee Implementation
Let’s talk about how you hire your AI employee. As I mentioned before, AI employees are hired rather than integrated – they don’t require changes to your existing software or business processes, and you don’t need to train your human employees.
Implementation Process: Five Simple Steps
Step 1: Finding Your Business
It’s very straightforward – you’ll see a field where you can enter your business URL or find your business on Google Maps. Either option works perfectly fine.
Step 2: Automated Learning Phase
This is where the magic happens! Once you click “train,” our creator platform starts learning about your business from public information. It’s fascinating to watch – in about 100-120 seconds, it:
- Understands your business name
- Identifies if you’re single or multi-location
- Captures physical address and working hours
- Learns about your products and services
- Analyzes everything available on your website or Google Maps
Step 3: AI Employee Creation
Now it gets really interesting. You’ll enter your basic information – email, phone number, and name. Whether you’re the business owner creating an AI receptionist for yourself or a service partner creating it for a client, this information helps our platform:
- Create your account
- Generate a complete multi-agent system
- Set up parallel flows, thinking processes, controllers, and auditors
- Establish control mechanisms for optimizing speed and reducing latency
The system essentially creates itself in about another 100 seconds!
Step 4: Initial Testing
Here’s where it gets exciting – you’ll receive a phone number, ideally in your business’s area code. This is your moment of truth! Simply pick up your phone and call this number, pretending to be a customer. The AI adapts based on your industry:
For a restaurant, your AI host will:
- Ask about table booking times
- Check the number of people
- Search for open slots
- Actually book the table!
For a cleaning service, it will discuss:
- Frequency (weekly, bi-weekly)
- Number of bedrooms and bathrooms
- Provide estimates
For medical spas or orthodontics, you’ll get a specialized AI receptionist familiar with those services.
Even for industries we haven’t specifically tailored, you’ll get a capable generic AI receptionist who can describe your services and handle appointment setting.
Step 5: Final Setup with Human Expert
This is where we perfect everything. You’ll meet with our human AI expert, who will:
- Help with final adjustments
- Set up service area specifications (like HVAC zip codes)
- Handle integrations with your service management systems
- Configure call transfer options (non-working hours, overflow calls, or full transfer)
Cost Structure
The pricing is quite straightforward and consumption-based:
- One unit = one 3-minute call
- Small plan: $99/month for up to 60 calls/month
- Corporate plan: More economical for 600+ calls/month
Subscription options:
- Annual commitment: $99/month
- No commitment: $119/month with freedom to cancel anytime
- Special promotions: Watch for free trial periods and onboarding bonuses!
You can make changes yourself or work with an AI professional for future modifications. Step by step, your AI employee will become an invaluable member of your human business team.