Developing applications with LLM APIs like OpenAI's is incredibly fast, with a functional prototype achievable in hours, not weeks.
Effective prompt engineering is a nuanced skill; prompts must be carefully balanced, as being too simple or too specific leads to model failure.
Fine-tuning model parameters such as temperature, frequency penalty, and presence penalty is critical for controlling the creativity, coherence, and quality of AI-generated text.
Out-of-the-box LLMs are unreliable for production use without human oversight, frequently fabricating information and producing low-quality content that requires significant manual correction.
There are significant ethical concerns in the AI space, including the practice of overcharging for simple AI wrappers and the potential for AI-generated content to flood and degrade systems like the job market.
Project Inception
Teng initiated her first LLM project, a resume cover letter generator, using OpenAI's GPT-3 API and the $18 in free credits provided to new users.
Initial Development
She experienced a surprisingly quick ramp-up time, creating a functional application with only a few lines of code, utilizing the OpenAI Playground for code-free prompt testing.
Model Selection and Prompt Engineering
Teng selected the DaVinci model for its larger token limit and more recent training data. She iteratively refined her prompts, discovering that both overly simplistic and overly specific inputs resulted in poor outputs.
Parameter Tuning
She adjusted parameters like 'temperature', 'frequency penalty', and 'presence penalty' to improve output quality, identifying a 'danger zone' where a high temperature combined with a long text request led to incoherent results.
Deployment and Performance Review
The application was deployed on Streamlit, handling 50-100 daily queries for under $2 in API costs over a month. Teng concluded that only 1 in 10 generated letters were 'sendable' after manual tweaking.
Broader Reflection
Following the project, Teng articulated ethical concerns about overpriced AI wrapper apps and the potential for AI-generated spam to disrupt the job market.
▶Rapid Prototyping and Accessibility of LLMs
Teng's experience building a cover letter generator in just five hours highlights the dramatically reduced barrier to entry for creating AI-powered applications. The availability of tools like the OpenAI Playground and simple APIs enables developers to move from idea to a deployed prototype with unprecedented speed.
This accessibility suggests a future proliferation of niche, single-purpose AI tools, but also indicates that sustainable competitive advantage will likely come from proprietary data, unique user experience, or sophisticated backend processing rather than simply wrapping a public API.
▶The Nuance and Necessity of Prompt EngineeringMay 2026
Teng details the delicate balance required for effective prompting, where overly simple prompts lead to generic falsehoods and overly specific ones cause the model to copy input verbatim. Her work underscores that interacting with LLMs is a nuanced skill involving iterative testing and precise parameter tuning to guide the model's output.
Prompt engineering is emerging as a critical human-in-the-loop skill, meaning the value of AI tools is heavily dependent on the operator's ability to effectively communicate context and constraints to the model. This creates a new layer of technical expertise that companies must cultivate.
▶Practical Limitations and Unreliability of LLMsMay 2026
Despite the technology's power, Teng's project revealed significant flaws in GPT-3. The model frequently produced factual errors, assigned incorrect attributes to users, and generated incoherent text, with a low success rate of only one in ten outputs being 'sendable' even after manual edits.
Analysts should remain skeptical of claims of full automation, as the 'last mile' problem of ensuring factual accuracy and quality control remains a major hurdle. The high rate of failure suggests that for many use cases, LLMs are better suited as assistants that augment human work rather than fully replacing it.
▶Emerging Business Ethics and Market DisruptionMay 2026
Teng raises ethical questions about the business models built on LLMs, specifically criticizing companies that charge high prices for simple API wrappers. She also speculates on the disruptive potential of the technology to flood systems like the job market with low-quality, AI-generated applications, creating new challenges for recruiters.
The low marginal cost of AI generation will likely lead to both market saturation and malicious use cases. This will force platforms and industries to develop new methods for detecting automated content and will create a market demand for services that can verify authenticity and quality.