Evalium Research

Evaluation is a research problem before it is an operations problem.

Our work on AI-assisted software engineering is published at ICSE, ESEC/FSE, and MSR — and it is the foundation of how we design evals, data, and environments.

Selected Publications & Preprints

AI code review & evaluation

Automated Code Review in Practice

Cihan, Haratian, İçöz, Gül, Devran, Bayendur, Uçar, et al.

What happens when automated code review meets real industrial pipelines — adoption, impact, and failure patterns in practice.

ICSE-SEIP 2025

Evaluating Large Language Models for Code Review

Cihan, İçöz, Haratian, Tüzün

A systematic evaluation of frontier LLMs as code reviewers: where they match expert judgment, and where they break.

Preprint 2025 · arXiv:2505.20206

Rethinking Code Review in the Age of AI: A Vision for Agentic Code Review

Kamalı, Tuna, Haratian, Tüzün

A forward vision for code review as an agentic, multi-step process — and what it demands from training data and evaluation.

Preprint 2026 · arXiv:2605.17548

Code & repository understanding

PR-Aware Automated Unit Test Generation: Challenges and Opportunities

Haratian, Akar, Çakar, Tüzün

Generating tests that understand the change, not just the code — groundwork for verifier design in SWE environments.

Preprint 2026 · arXiv:2605.25285

RefExpo: Unveiling Software Project Structures through Advanced Dependency Graph Extraction

Haratian, Derakhshanfar, Kovalenko, Tüzün

Extracting reliable dependency graphs at repository scale — infrastructure for repo-level task design.

MSR 2025

BFSig: Leveraging File Significance in Bus Factor Estimation

Haratian, Evtikhiev, Derakhshanfar, Tüzün, Kovalenko

Which files actually matter in a codebase — a signal we reuse when selecting repositories and tasks for coding data.

ESEC/FSE 2023

Full publication list on Google Scholar →

Applied work

Coming from the lab.

We're preparing our first public artifacts.

Our team serves as a reviewer at MSR and works actively within the software engineering research community.

Want early access when these ship?

Reach out and we'll keep you in the loop as our first public benchmarks and posts go live.

Get in touch