Jan 16, 2025
don't you think this method will makes sense only when we know token cost will drop significantly. otherwise going with the other approaches makes more sense and less complexity? what do you think about it? I can see people using that same recursive character approach in production too , not just because of simplicity but because it worked too. do you think we'll see people using llm to chunk?,