Faulty AI that told people to break the law defended by New York mayor

AI
(Image credit: Shutterstock)

A AI chatbot recently introduced to the city of New York as a way for business owners to get 24/7 access to "actionable and trusted information” has been defended by Mayor Eric Adams after it was called out for encouraging entrepreneurs to break the law.

The MyCity AI chatbot uses Microsoft’s Azure AI large language models, but is plagued by the same issues that most AI models suffer from, including giving wrong and misleading information.

In response to the criticisms, Mayor Adams stated that the AI tool is “wrong in some areas, and we've got to fix it,” further defending the chatbot saying that, “Any time you use technology, you need to put it into the real environment to iron out the kinks.”

The chatbot was previously reported as telling landlords that they were free to discriminate based on income, and that business owners could take workers tips as their own - despite both of these practices being illegal in New York.

Subsequent probes of the chatbot’s responses on Thursday revealed that it was still giving out incorrect information, with Reuters reporting that the AI had said that stores no longer had to accept cash as a method of payment, despite the fact that this violates New York law.

The director of the NYC Hospitality Alliance, Andrew Rigie, said “I commend the city for trying to use AI to help businesses, but it needs to work. “If when I ask a question and then I have to go back to my lawyers to know whether or not the answer is correct, it defeats the purpose.”

The MyCity chatbot has since been updated to state that the AI’s responses “may sometimes be inaccurate or incomplete” and that the answers given in response to queries should not be used as legal or professional advice.

More from TechRadar Pro

Benedict Collins
Staff Writer (Security)

Benedict has been writing about security issues for close to 5 years, at first covering geopolitics and international relations while at the University of Buckingham. During this time he studied BA Politics with Journalism, for which he received a second-class honours (upper division). Benedict then continued his studies at a postgraduate level and achieved a distinction in MA Security, Intelligence and Diplomacy. Benedict transitioned his security interests towards cybersecurity upon joining TechRadar Pro as a Staff Writer, focusing on state-sponsored threat actors, malware, social engineering, and national security. Benedict is also an expert on B2B security products, including firewalls, antivirus, endpoint security, and password management.

TOPICS