Automated VFX Audit (UE5, FT, Niagara, Jenkins)

Unreal Engine, Niagara, Functional Test, Python, Jenkins

Introduction

While at Imporium, the process of checking overdraw and instruction counts was largely manual, a reactive approach that often led to late stage optimization scrambles. Realizing this gap, I decided to spend my own time developing a fully automated VFX Performance Audit Pipeline to turn performance monitoring from a manual chore into a continuous, data-driven workflow.

The Problem: Manual Auditing Doesn’t Scale

VFX performance is notoriously volatile. A single change in a Niagara emitter’s spawn rate or a shader’s transparency can tank a frame rate. Checking these daily requires:

  1. Opening every individual asset.
  2. Switching viewmodes to Shader Complexity/Overdraw.
  3. Recording Render Thread timings manually.

In a fast-paced environment, this is rarely sustainable. I wanted a system where I could simply check a Jenkins dashboard and see exactly which assets exceeded our 0.5ms budget (the value can be adjusted).

The Solution: An Automated Pipeline

I built a 3 stages pipeline that integrates Unreal Engine, Python, and Jenkins to provide immediate visual and technical feedback.

0. Automated Asset Discovery (Editor Utility Widget)

I created this Editor Utility Widget to get all VFX asset to be tested. It collect all NS in the project (ignoring engine content), then check if they are already exist and populate the data table with reference to it.

  • Registry-Based Discovery: Uses the Unreal Asset Registry to automatically scan the project’s /Game directory for all NiagaraSystem assets, filtering out engine-level content.
  • Delta Sync Logic: Performs a “Check and Skip” operation that compares the current project folder state against the existing Data Table. This preserves manual overrides and custom performance metadata while only appending newly discovered effects.
  • Path Sanitization: Formats asset references into a CSV-compliant string with double-quoted paths to prevent parsing errors during the headless automation phase.
  • Pipeline Readiness: Ensures the Data Table, which acts as the “Source of Truth” for the Functional Test is always up-to-date, transforming the audit into a zero-maintenance workflow.

1. Unreal Engine Automation (Functional Test)

The pipeline starts with a Functional Test in UE5 on a Data Table containing all Niagara System to be tested. Using a headless runner (-RenderOffScreen), the engine automatically:

  • Spawns each Niagara system in an empty level.
  • Triggers the CSV Profiler to capture exclusive Niagara Render Thread timings.
  • Toggles the viewmode ShaderComplexity and captures high-resolution “burst” screenshots to provide visual proof of overdraw.

2. Parsing the data (Python)

https://gist.github.com/aobond2/31e852ad046698c3500d6f3a600c2fb7

Raw data is messy. I developed a Python script to parses the timestamped CSV logs, identifies the specific asset tested via metadata events, and calculates the average performance metrics. Crucially, it matches the correct overdraw heatmaps to each asset entry, consolidating everything into a single, clean results folder.

3. Visualization: Jenkins & HTML Reporting

The final output is a responsive HTML report hosted on Jenkins. Instead of digging through raw data, the team receives a clear table showing:

  • Average Niagara Render Thread time.
  • Pass/Fail status based on our performance budget.
  • A visual gallery of overdraw heatmaps. (WIP)
  • Direct links to .utrace files for deep-dive analysis in Unreal Insights.

Key Maintenance: Clean Data, Clean Drive

To ensure the pipeline is production-ready, I implemented a retention policy within the Python logic. The script automatically prunes results and trace folders older than 7 days, preventing the profiling server from running out of storage while maintaining a rolling window of recent audit history.

Conclusion

By automating the “boring” parts of VFX management, we free up artists to do what they do best: create. This pipeline ensures that our creative vision never comes at the cost of technical stability.

This doesn’t replace profiling pass needed on VFX, this is meant to be help as early gate to catch expensive VFX.