Research

S-LakeNLP — Streaming Lakehouse with Embedded NLP

M.Tech research: streaming data architecture combining Kafka, Flink, FinBERT inference and FAISS vector search, evaluated for latency, resource usage and model performance.

GitHub

Role: M.Tech research project
Year: 2026
Status: Research

Challenges & solutions

01
Keep NLP inference latency low inside a stream
performanceresearch
Problem
Transformer inference is expensive per event.
Solution
Parallel Flink operators and batched inference; measured end-to-end latency.

Deep dive

🌊 S-LakeNLP

Streaming Lakehouse with Embedded NLP — M.Tech research project (May 2026).

Overview

A streaming data architecture combining Apache Kafka, Apache Flink, FinBERT inference and FAISS vector search.

Real-time processing, NLP inference and vector retrieval in a distributed data pipeline
Evaluated processing latency, resource usage and model performance for the proposed architecture

Architecture

Kafka topics ──▶ Flink stream processing ──▶ FinBERT sentiment inference
                                                   │
                                   Lakehouse storage ◀┘──▶ FAISS vector search

Next project

Portfolio — this website