Back to blog

G-Research tackles the biggest financial data challenges of tomorrow with Polars

Thu, 18 Apr 2024

G-Research banner

About G-Research

G-Research is a leading quantitative finance research firm, with offices in London (UK) and Dallas (USA). We tackle the biggest questions in finance and pair this expertise with machine learning, big data and emerging tech to predict movements in financial markets.

Data is at the heart of what we do - multi-exa-loads of it, in all shapes and sizes. Finding cutting-edge tools that make handling data less of a headache is always good news at G-Research.

Enter Polars: not just another library but a revolution in dealing with data more efficiently and with less fuss in our company.

What Makes Polars So Special for G-Research?

Speed That Speaks Volumes

Teams at G-Research are genuinely excited about how much quicker Polars is, especially when dealing with gargantuan group bys, complex strings or juxtaposed joins that could slow down workflows to a crawl. One user pointed out he helped a colleague to get a 150x speedup over our previous implementation. That’s not just an improvement for G-Research; it’s a game-changer.

The speedup of Polars compared to pandas is massively noticeable. I generally enjoy writing code that I know is fast.

Machine Learning Engineer @ G-Research

The Syntax Just Makes Sense

Polars syntax is like a breath of fresh air, making code not only look cleaner but actually easier to follow and maintain. No more wrestling with weird index columns after grouping your data – Polars keeps it clean and simple.

Quick Fixes and Helpful Tips

Ever found a bug in a tool and felt like you were shouting into the void? Our users tell a different story. The Polars development team and community are redefining the meaning of being responsive to users. “I reported a bug and it was fixed within a day. Very impressive!” says one of our engineers. Plus, the library is full of helpful error messages that guide you on how to make your code better.

What Users at G-Research Are Saying

The impact of Polars is noticeable throughout the organization as more teams are starting to use Polars in their daily work. Here are a couple of quotes from machine learning engineers and researchers at G-Research on their experience working with Polars in their projects:

We picked Polars because it is faster and more memory efficient, gives a better experience, and has a much cleaner API compared to pandas, especially for expressions.

Our team saw a 20x speedup in dequantization when switching from pandas to Polars. Polars' fast multi-threaded joins took advantage of all of the cores available on the machine whereas pandas was only able to run on a single core.

The internal plotting library is built with Polars, which is used by researchers to visualize different metrics they log. Compared to Pandas, we’re using significantly less memory than before without sacrificing the speed of generating user visualizations.

The Bottom Line: Polars is Changing the Game

Polars isn’t just another tool in our shed; it’s shaping up to be one of the definitive tools for data processing. With its speed, user-friendly approach, and a community that’s actively supported by the developers, it’s exciting to think about how Polars will continue to evolve and help us tackle the biggest financial data challenges of tomorrow.

Find out more about G-Research here: https://www.gresearch.com/about/about-us/. G-Research is always on the lookout to tackle the biggest financial data challenges of tomorrow. Learn more about open positions here: https://www.gresearch.com/vacancies/

1
2
4
3
5
6
7
8
9
10
11
12