I thought that I knew a thing or two about sql indexes. It’s easy right, you just run a couple sql commands to create your index, and you’re off to the races. Turns out that it isn’t quite that easy to get the index you need. You need to know a few things.
If you’re brand new to sql and have never heard of indexes, it would make sense that we take a bit of a sidebar and discuss why they are important. Let’s lay a bit of groundwork to get us started on this discussion.
Curious how you can get started processing 100 Million data points, using only 150 MiBs of memory, from a native GUI application? Keep reading to figure out how we’ve been working on this problem.
In a previous post, we talked about the design goals that were important to a rainflow cycle counting application that we’re working on. Two of those goals are in somewhat of a conflict, Efficient and Quick, but efficient edged out over quick in the priority scale. We decided being able to run on lower spec hardware instead of a massive datacenter machine with gobs of memory, was more important than being super quick.
At High Creek Software we have been working on a purpose specific tool to extract cycles out of an inline inspection dataset. Our hope is to provide an accurate and fast process for extracting cycle data to be used in fatigue life analysis.
To get started, this is still a work in progress, but we feel that we’ve made good enough progress to start talking about it. Right now, we have a project based UI where you can: