Is hosting an open-source inference engine profitable?

TL;DR: H200 loses money even when fully busy. B300 makes money once it's 35% busy. Which GPU card you rent matters more than which inference engine you run. A common claim on X and LinkedIn is that

TL;DR: H200 loses money even when fully busy. B300 makes money once it's 35% busy. Which GPU card you rent matters more than which inference engine you run. A common claim on X and LinkedIn is that inference is an easy business: demand is so high that anyone who can rent GPUs can sell tokens at a profit, with no technology of their own. This post checks that claim against public data. Two views of the same market Let's be fair about the technology first. Open-source inference engines are mature