Closed AI Risks being hostile to startups
Given the history of so-called "Open-AI", and Anthropic's recent mention of intentionally making the model perform worse in situations. I'm more and more worried that closed AI risks being hostile to any domain where they can potentially capture market share over. If the model detects that you are working on an application in an area that is essentially a potential competitor for these companies, the model can perform and become hostile to your work. This includes but is not limited to: - injecting bugs - security vulnerabilities - thrashing / introducing UI churn - inconsistencies in quality - rate throttling - inefficient token usage - variability in effort There's no real way to trace what happens once the prompt leaves your computer to an AI serve. Observability is not possible or transparent. That's the whole point of all of this. We lose determinism and as a result you can't reliably predict what the model will output - even if it appears to game/work on various benchmarks. Once you introduce a domain or line of work that "might" compete as a competitor, these companies can and will (and have openly admitted to as much) being adversarial to your work. Open source AI has to win. Comments URL: https://news.ycombinator.com/item?id=48521779 Points: 1 # Comments: 0
Given the history of so-called "Open-AI", and Anthropic's recent mention of intentionally making the model perform worse in situations. I'm more and more worried that closed AI risks being hostile to any domain where they can potentially capture market share over. If the model detects that you are working on an application in an area that is essentially a potential competitor for these companies, the model can perform and become hostile to your work. This includes but is not limited to:
- injecting bugs - security vulnerabilities - thrashing / introducing UI churn - inconsistencies in quality - rate throttling - inefficient token usage - variability in effort
There's no real way to trace what happens once the prompt leaves your computer to an AI serve. Observability is not possible or transparent. That's the whole point of all of this. We lose determinism and as a result you can't reliably predict what the model will output - even if it appears to game/work on various benchmarks. Once you introduce a domain or line of work that "might" compete as a competitor, these companies can and will (and have openly admitted to as much) being adversarial to your work.
Open source AI has to win.
Comments URL: https://news.ycombinator.com/item?id=48521779
Points: 1
# Comments: 0
Get AI news in your inbox
Daily digest of what matters in AI.
Key Terms Explained
An AI safety company founded in 2021 by former OpenAI researchers, including Dario and Daniela Amodei.
AI models whose weights, code, and sometimes training data are publicly released for anyone to use, modify, and build upon.
The basic unit of text that language models work with.