Fully Connected Tokyo• Jan 23, 2026• 2:24:28

Fully Connected Tokyo: [Hands-on workshop] Building and improving customer support AI

From Fully Connected Tokyo

Fully Connected Tokyo

Executive Summary

The discussion provides a technical overview of building advanced customer support AI applications using the LangChain framework.
It emphasizes a modern architecture that includes AI agents, function calling, and semantic search to move beyond simple FAQ chatbots.
A multi-provider strategy for large language models (OpenAI, Google, Anthropic) is recommended for resilience and optimization.
The importance of a continuous feedback loop is highlighted, utilizing tools like Weave for human annotation, tracing, and evaluation to improve system performance.

6 quotes

Concerns Raised

Managing and mitigating LLM hallucinations.
Ensuring consistent quality and accuracy of AI responses.
The complexity of building and maintaining stateful, multi-turn applications.

Opportunities Identified

Significant cost reduction and efficiency gains in customer support operations.
Automating complex, end-to-end business workflows by integrating AI agents with RPA.
Creating highly personalized and context-aware customer experiences.

Key Themes

AI-Powered Customer Support Automation

The core focus is on leveraging AI to automate and enhance customer support workflows. This involves building sophisticated chatbots and AI agents capable of understanding context, performing actions, and resolving complex queries, thereby reducing the burden on human agents.

This theme is critical for businesses looking to improve customer service efficiency, reduce operational costs, and provide 24/7 support.

The LangChain Ecosystem for Development

The episode details the use of the LangChain framework and its components, like LangGraph, as the foundation for building these AI applications. It provides the structure for chaining LLM calls, managing state, and integrating various tools and data sources.

For developers and architects, understanding this ecosystem is key to rapidly building, prototyping, and deploying complex, stateful AI-driven applications without starting from scratch.

Multi-Provider LLM Strategy

A key strategic recommendation is to avoid vendor lock-in by using multiple LLM providers (e.g., OpenAI, Google Gemini, Anthropic). This approach allows for selecting the best model for a specific task, optimizing for cost and performance, and ensuring resilience if one provider experiences an outage.

This strategy provides businesses with greater flexibility, cost control, and operational stability in their AI deployments, mitigating the risks associated with dependency on a single technology provider.

MLOps for LLMs: Evaluation and Feedback Loops

The discussion underscores that building an AI agent is not a one-time task; it requires continuous monitoring and improvement. It covers the necessity of robust evaluation, tracing, and a human-in-the-loop feedback process, using tools like Weave to annotate responses and refine the system's accuracy and reliability.

This addresses the critical 'last mile' of AI deployment, ensuring that applications perform well in production, adapt over time, and maintain a high standard of quality and user trust.

Get started free

Topics

LangChain AI Agents Customer Support Automation Chatbots Large Language Models (LLMs)Multi-LLM Strategy LangGraph Function Calling Semantic Search Retrieval-Augmented Generation (RAG)MLOps for LLMs AI Evaluation Human-in-the-Loop (HITL)Robotic Process Automation (RPA)AI Application Development

Processed Apr 9, 2026 yt-dlp + mlx-whisper + Gemini