yea, but no prompt caching right? This makes it unusable for my usecase at least, the cost would be insane
They confirmed on LinkedIn that they are working on it.
Also, they have a feature request that has been getting many votes recently: https://feature-request.scaleway.com/posts/1251/prompt-cachi...
alt Hacker News
They confirmed on LinkedIn that they are working on it.
Also, they have a feature request that has been getting many votes recently: https://feature-request.scaleway.com/posts/1251/prompt-cachi...