UK spy agency warns of security threat from ChatGPT and rival chatbots

A UK intelligence agency has warned of potential security threats from ChatGPT and other large language model-based chatbots, recommending users do not enter personal or sensitive information into the software.

Writing on the National Cyber Security Centre’s blog, two tech directors – David C and Paul J – highlighted the main areas of concern – privacy leaks and use by cyber criminals.

Large language models are trained on huge datasets (e.g. the whole internet), but once that information is ingested, they do not continue to learn from the prompts entered by users – of which ChatGPT receives millions per day.

However, while that means at present there is no risk of the chatbot repeating a user query as part of an answer to someone else, all of the queries are stored by its developers. One day, the agency argues, developers could use these saved queries to further develop the LLM model.

As an example, the NCSC team writes: ‘A question might be sensitive because of data included in the query, or because who is asking the question (and when). Examples of the latter might be if a CEO is discovered to have asked ‘how best to lay off an employee?’, or somebody asking revealing health or relationship questions. 

‘Also bear in mind aggregation of information across multiple queries using the same login.’

In addition to later use by the developer, queries could also be hacked, leaked or accidentally made public. While web browsers typically store search histories and are vulnerable to similar situations, users can clear their previous searches.

NCSC advice for safely using LLM chatbots

  • Do not include sensitive information in queries to public LLMs
  • Do not submit queries to public LLMs that would lead to issues were they made public

In the case of more malicious use, the NCSC also suggests LLM chatbots will be capable of helping hackers or scammers create more convincing phishing emails – in multiple languages – and assist attackers in writing more advanced malware than they have previously attempted. 

The team also flagged a low risk of LLMs enabling less skilled attackers to write highly capable malware.

On the flaws in current generative artificial intelligence LLMs – as opposed to artificial general intelligence, the Holy Grail for developers – the NCSC highlighted a number of issues, including the bots ‘getting things wrong and ‘hallucinating’ incorrect facts’, showing bias and being gullible, and being ‘coaxed into creating toxic content’.

However, it adds: ‘LLMs are undoubtedly impressive for their ability to generate a huge range of convincing content in multiple human and computer languages.’

Since its launch in November last year, OpenAI’s ChatGPT has been used by millions of people across the globe, being adopted by schoolchildren and businesses, helping with homework and fighting speeding tickets.

Microsoft has invested $10billion in OpenAI, and incorporated ChatGPT into its search engine Bing. Its Edge web browser will soon include a ChatGPT sidebar.

In February, Google launched its own LLM chatbot, Bard, to much fanfare. However, after the software generated a wrong answer during its first public display, more than $100billion was wiped off the company’s market value. Bard incorrectly stated that the James Webb Space Telescope was the first to photograph a planet outside our solar system.

Meta’s own chatbot LLaMA was only intended for use by those in the AI community, but shortly after its release last month, it was leaked. However, the company is also working on a public chatbot.

Source: Read Full Article