Artificial Intelligence Specialist

NCS is a leading AI Tech Services company. With a 15,000-strong team across the Asia Pacific, NCS scales its platforms and capabilities to provide clients with greater agility and AI expertise across a range of Industries. Embracing a strong ecosystem of global partners, NCS transforms technology services delivery combining AI with digital resilience to drive real business impact. NCS is a subsidiary of the Singtel Group.

This role is an

AI/LLM Specialist focused on prompt engineering, model integration, evaluation, and production quality

within NCS AI Central's Forward Deployed Engineering model. This role helps take GenAI solutions from POC/POV to production by building prompts, integrating foundation models, creating automated evaluation and benchmarking frameworks, and continuously monitoring quality, safety, hallucination, bias, drift, latency, and cost.

You will work closely with AI Architects, Solution Architects, AI Engineers, and Testers to provide evidence-based model recommendations, support production readiness, and maintain reusable internal assets such as prompt libraries, evaluation templates, and benchmark datasets.

What will you do

1. Model Integration & Prompt Engineering

Design, test, and optimise prompts and prompt chains for production use cases — balancing accuracy, latency, and cost.

Integrate foundation models into applications via APIs and gateways; advise on model/version selection for a given use case alongside AI Architects.

Support light fine-tuning and instruction-tuning work (LoRA/PEFT and similar techniques) where a use case calls for it, in partnership with AI Engineers.

2. Evaluation Framework & Benchmarking

Design and maintain evaluation harnesses (golden datasets, benchmark suites) that measure LLM/agentic system accuracy, consistency, and safety.

Run comparative model benchmarking (accuracy, latency, cost-per-query) to feed technical evidence into model-selection decisions led by AI/Solution Architects.

Build repeatable, automated regression suites that run on every prompt, model, or pipeline change, wired into CI.

3. Quality & Adversarial Testing

Proactively red-team AI systems — adversarial prompting, edge-case and jailbreak testing — to surface failure modes before clients do.

Detect and quantify hallucination, bias, and drift in production and pre-production systems, producing clear, defensible metrics (not qualitative impressions).

4. Governance & Reporting

Feed evaluation evidence into the PRR (Production Readiness Review) Evaluation & Quality pillar, supporting engagement teams at the Scale gate.

Maintain evaluation and prompt-version documentation and reporting standards that align with client compliance and audit needs (e.g., government AI governance requirements).

5. FDE & Development/Maintenance Coverage

During FDE engagements: rapidly prototype prompts and model integrations, and stand up lightweight evaluation harnesses to compare candidate models/approaches during POC/POV, giving the team fast, evidence-based go/no-go signals.

During system development & maintenance engagements: own ongoing prompt/model tuning and run continuous evaluation and regression monitoring on live production systems, flagging quality degradation over time.

Contribute reusable prompt libraries, evaluation templates, and benchmark datasets back into the shared internal asset library for reuse across engagements.

6. Collaboration

Provide technical benchmark evidence and integration recommendations to AI Architects and Solution Architects, who own the final client-facing model recommendation and proposal.

Partner with AI Engineers and Testers to distinguish functional QA (does it work) from output-quality evaluation (is it right), and to hand off tuned prompts/models cleanly into production builds.

The ideal candidate should possess:

3+ years working hands-on with LLMs across prompt engineering, model integration, and evaluation — not evaluation alone.

Practical experience designing and optimising prompts and prompt chains for production applications, and integrating models via APIs/gateways.

Strong grasp of evaluation methodologies — accuracy/hallucination/toxicity metrics, human-in-the-loop evaluation, A/B testing.

Hands-on scripting ability (Python) to build and automate evaluation harnesses and integration/testing pipelines.

Statistical literacy — able to design a representative test/benchmark set and interpret results rigorously, not anecdotally.

Clear, structured written communication — able to translate evaluation results and model recommendations into a defensible report for both engineering and client audiences.

Working knowledge of the China AI model/tech stack (e.g., DeepSeek, Qwen, GLM, Kimi, MiniMax) — deployment patterns, licensing, and self-hosting requirements.

Preferred Qualifications

Hands-on fine-tuning/instruction-tuning experience (LoRA/PEFT or similar) on open-weight models.

Experience with LLM evaluation tooling (RAGAS, DeepEval, TruLens, promptfoo) or building custom eval frameworks.

Exposure to red-teaming/adversarial testing practices for generative AI systems.

Familiarity with regulated-sector AI governance expectations (Healthcare, Government, Financial Services).

Prior experience supporting client-facing presales or solutioning conversations with technical evidence (without owning the proposal).

Hands-on benchmarking or integration experience with Chinese open-weight models (DeepSeek, Qwen, GLM) alongside Western models.

Tech Stack (Illustrative)

Languages: Python (primary), SQL

Prompt & Integration: LangChain/LlamaIndex, model gateways (LiteLLM, Bedrock, Azure OpenAI), prompt-versioning tools

Fine-Tuning: LoRA/PEFT, Hugging Face Transformers (where applicable)

Eval Tooling: RAGAS, DeepEval, TruLens, promptfoo, custom harnesses

LLM Runtime: OpenAI/Azure OpenAI/Bedrock/Vertex APIs; DeepSeek/Qwen/GLM (China stack)

Data & Reporting: Pandas, Jupyter, BI/reporting tools for evaluation dashboards

CI Integration: GitHub Actions/GitLab CI for automated regression evaluation

Why Join NCS

Grow with Us

Work on cutting-edge AI products that shape the future of technology

Collaborate with talented, passionate teams across research, engineering, and design

Access continuous learning opportunities and career development pathways

Make an Impact

Transform AI research into products that solve real problems for clients and users

Drive innovation in a leading Technology Services Firm with regional presence

Contribute to building a better future through responsible, human-centred AI

Thrive in Our Culture

Experience a

human-to-human

approach where relationships and collaboration matter

Be part of

Team NCS

, where bold ideas meet practical execution

Enjoy a supportive environment that values diversity, inclusion, and respect

We are driven by our AEIOU beliefs—Adventure, Excellence, Integrity, Ownership, and Unity—and we seek individuals who embody these values in both their professional and personal lives. We are committed to our Impact: Valuing our clients, Growing our people, and Creating our future.

Together, we make the extraordinary happen.

Learn more about us at ncs.co and visit our LinkedIn career site.