alt Hacker NewsThey are very good research assistants.
I would never trust an AI to make a scientific conclusion, but they're very good at some things:
- Plotting with natural language: "Create a scatter plot of column A versus column B, log scale Y and use stars sized by column C." Every plotting library (except maybe ggplot2 / plotnind) sucks, and the boilerplate never gets less tedius.
- Literature search, e.g. "Have there been any peer reviewed papers where they applied X method to Y domain"
- Complex visualization, e.g. "Make a self contained HTML file with a 3D interactive visualization of this graph. Add ball and spring position optimization and let me move nodes"
- Trying random stuff that would normally be too time consuming to be worth it, eg "Here is a dataset. Try regression methods A, B, and C and report a table." In this case, usually it will screw up and overfit, but it's a good smoke test.
Note how in all of these, it's being used for tedious things that provide some value, but ultimately they are only providing information to make a decision on.
Granted, I would like to have a research assistant I could train, but if your company will not create a position then there is not much you can do.
Replies
So you're not wrong in most of these conclusions, however this is very surface level use case for these tools when it comes to scientific research.
> I would never trust an AI to make a scientific conclusion, but they're very good at some things
> In this case, usually it will screw up and overfit, but it's a good smoke test.
> it's being used for tedious things that provide some value
I see a lot of comments like this about AI use in general, and I suspect there’s a kind of learned helplessness combined with anchoring to earlier experiences. Someone tries a model, encounters a few failures, and forms a fairly permanent view of what these systems can do.
My concern is that people sometimes turn an expectation of failure into a permanent ceiling on how much they’ll even experiment with these tools. Their limited use then keeps confirming their limited expectations.
EDIT: This came off as criticizing the user, but this is entirely to validate my own thesis that for quite a long time nobody has been able to crack the usability challenge when it comes to non-coding knowledge work for these harnesses. It's fascinating because it seems like it's such an easy problem to solve, but in reality it's a complex product problem.
> Literature search, e.g. "Have there been any peer reviewed papers where they applied X method to Y domain"
This is something the LLMs are really good for, at least for compiler papers.
They are great computer interface assistants, but forget scientific conclusion would never trust then for committing in right git branch
Yes to all but I have trouble using them effectively for my research side projects or Kaggle competitions. Claude Code/Codex designs and runs experiments quickly, invalidates my initial idea using some obscure criteria than changes direction, and after long hours and days of experiments we end up with poor results. There must be a better agentic workflow for research.