Weekly Digest on AI and Emerging Technologies (13 January 2024)

HIGHLIGHTS FROM DAILY DIGEST – WEEK 6 TO 10 JANUARY 2025

 

Governance and Legislation

 

Rogue AI Moves Three Steps Closer

(Peter N. Salib – Lawfare – 9 January 2025) Will humans lose control over advanced AI systems? Three developments from the final weeks of 2024 should make us worry. In short: Two empirical evaluations showed that systems like GPT-4 and Claude sometimes actively resist human efforts to alter their behavior. Those AIs try to resist human control by lying, faking compliance, disabling oversight mechanisms, and even copying themselves to external servers. Claude and GPT-4, however, are not yet smart enough to succeed at subverting human control. But OpenAI’s new o3 model—unveiled mid-December but not yet available to the public—suggests that it will not be long before AI systems are as smart as their human minders, or smarter. – https://www.lawfaremedia.org/article/rogue-ai-moves-three-steps-closer

Analysis: Meta’s fact-checking pullback will have global consequences

(Mark Scott – Atlantic Council – 8 January 2025) On January 7, 2025, Meta announced a wholesale change to how the company polices harmful, illegal and divisive content on Facebook and Instagram. The updates include the closure of its fact-checking program, starting in the US, that cuts off funding for independent groups to debunk falsehoods posted on its social media networks. – https://dfrlab.org/2025/01/08/analysis-metas-fact-checking-pullback-will-have-global-consequences/

 

The False Prophets of Silicon Valley

(Kyle Hiebert – Centre for International Governance Innovation – 8 January 2025) “In the future, everyone’s lives can be better than anyone’s life is now,” wrote OpenAI’s leader Sam Altman in a September blog post. It’s a spectacular assertion from a 39-year-old CEO rumoured to be gaining a seven percent equity stake in a company valued at US$157 billion. Altman insists artificial intelligence (AI) will soon outperform the mental abilities of most people. Afterward, he foresees climate change being solved, poverty erased and humanity colonizing space. All of this, Altman claims, is just around the corner — if only tech titans receive enough resources and leeway. Such views are common in Silicon Valley. Google DeepMind CEO and co-founder Demis Hassabis says software capable of human-level reasoning — artificial general intelligence, or AGI — will emerge by 2030. DeepMind’s chief AGI scientist, Shane Legg, pegs it at 2028. The head of Anthropic, Dario Amodei, sees humans lagging behind machines as early as 2026. Elon Musk thinks this could happen by the end of this year. – https://www.cigionline.org/articles/the-false-prophets-of-silicon-valley/

 

Saudi Arabia Lays Out Its Strategic Vision For The Quantum Era

(Matt Swayne – Quantum Insider – 6 January 2025) The Quantum Economy Landscape in Saudi Arabia report offers an in-depth analysis of how the Kingdom is is pursuing a leadership role in quantum technology as part of its Vision 2030 plan to diversify the economy and drive innovation. The Kingdom focuses on education, research, and partnerships to build a quantum ecosystem, highlighted by collaborations with global leaders like IBM and Pasqal. Key challenges include the need for more talent and limited infrastructure, but strategic plans emphasize education, infrastructure and international collaboration to bridge these gaps. – https://thequantuminsider.com/2025/01/06/saudi-arabia-lays-out-its-strategic-vision-for-the-quantum-era/

India Proposes Digital Data Rules with Tough Penalties and Cybersecurity Requirements

(Ravie Lakshmanan – The Hacker News – 6 January 2025) The Indian government has published a draft version of the Digital Personal Data Protection (DPDP) Rules for public consultation. “Data fiduciaries must provide clear and accessible information about how personal data is processed, enabling informed consent,” India’s Press Information Bureau (PIB) said in a statement released Sunday. “Citizens are empowered with rights to demand data erasure, appoint digital nominees, and access user-friendly mechanisms to manage their data.” – https://thehackernews.com/2025/01/india-proposes-digital-data-rules-with.html

AI will protect global supply chains from the next major shock

(Pervinder Johar – World Economic Forum – 5 January 2025) Climate change, geopolitics, and industrial action are still exerting pressure on global supply chains. Advanced technologies such as AI can help refine and optimize supply chains in 2025. Greater supply chain visibility, through initiatives such as GRSI, is an area ripe for technological transformation. – https://www.weforum.org/stories/2025/01/ai-supply-chains/

How AI can unlock new possibilities for global productivity and sustainability

(Roland Busch – World Economic Forum – 3 January 2025) In advanced economies, innovation rates and productivity growth have been declining for decades. We need a digital strategy to refresh the economic landscape and, with technologies like AI, we have one. Technologies, like AI and digital twins, combined with industry-specific know-how can make cutting-edge solutions more accessible and help us tackle our most pressing challenges, from equitable growth to sustainability. – https://www.weforum.org/stories/2025/01/tech-ai-digital-twins-productivity-sustainability/

Audits as Instruments of Principled AI Governance

(Anulekha Nandi – Observer Research Foundation – 3 January 2025) Stakeholder groups have produced various guidelines on ethical Artificial Intelligence (AI) in recent years. However, translating principles into practice continues to be a massive challenge, as AI markets expand and AI risks are heightened. AI audits—or the process of investigating an algorithm against existing regulations and known harms—are emerging as a way of bridging the gap between principle and practice. This paper scans the landscape of AI audits and highlights the roles of industry organisations and technical bodies, governments, civil society organisations, academics, and researchers as well as the necessity of developing procedural standardisation, the skillsets required for audit teams, and determining the appropriate nature of regulation and compliance mechanisms. – https://www.orfonline.org/research/audits-as-instruments-of-principled-ai-governance

Technology for Better Governance: Insights from Public Health Systems in Kenya

(Samir Bhattacharya, Israel Nyaburi Nyadera, Zedekia Opondo Sidha – Observer Research Foundation – 2 January 2025) The integration of technology in governance has created transformative possibilities across the world, enhancing the provision of public services. Africa has also benefitted from this wave of digital transformation. Introducing big data applications into governance systems in Africa presents enormous potential for overcoming challenges, including corruption, inefficiency, and poor service delivery. However, while the continent has made strides in digital governance, the full potential of big data is yet to be realised. For example, only a handful of African countries, such as Benin, Ghana, and Rwanda, have initiated e-government platforms, and with varying degrees of success. To reap the benefits of big data, there is a need to adopt digital governance in a more holistic manner, which includes leveraging artificial intelligence (AI), the Internet of Things (IoT), and smart city technologies. These innovations can improve policymaking, facilitate performance monitoring, and reduce corruption while enhancing the response time and quality to challenges such as urban migration, famine, and health crises. –  https://www.orfonline.org/research/technology-for-better-governance-insights-from-public-health-systems-in-kenya

 

Why AI Did Not Upend the Super Year of Elections

(Kevin Frazier – Lawfare – 2 January 2025) How AI labs and public policies helped safeguard the 2024 elections—and what to learn in order to protect democracy from future threats. – https://www.lawfaremedia.org/article/why-ai-did-not-upend-the-super-year-of-elections

Border Technologies Under Trump 2.0

(Petra Molnar – Just Security – 2 January 2025) The day after Donald Trump was reelected U.S. president, stocks in private prison firms soared. These firms may soon expand prisons to house detainees awaiting deportation and propel an increase in surveillance and artificial intelligence-driven technologies to accomplish this task. – https://www.justsecurity.org/105662/border-technologies-under-second-trump-administration/

Security

Government Launches £1.9m Initiative to Boost UK’s Cyber Resilience

(Phil Muncaster – Infosecurity Magazine – 9 January 2025) The UK government has provided more details of a new scheme designed to help the nation better protect itself from cyber-threats. It announced £1.9m ($2.3 m) in government and private sector funding for 30 “Cyber Local” projects across England and Northern Ireland, which it hopes will protect the digital economy and grow the UK’s cyber skills. – https://www.infosecurity-magazine.com/news/government-19m-boost-uks-cyber/

New AI Challenges Will Test CISOs & Their Teams in 2025

(Josh Lemos – Dark Reading – 9 January 2025) Security teams have always had to adapt to change, but new developments that will play out over the next year could make 2025 particularly challenging. The accelerating pace of AI innovation, increasingly sophisticated cyber threats, and new regulatory mandates will require chief information security officers (CISOs) to navigate a more complex landscape. Vendors are rapidly adding AI-enabled features to existing products, and the foundational large language models (LLMs) they are using present a new attack surface that malicious actors will try to exploit. CISOs will need to understand their level of exposure to these threats and how to mitigate them. – https://www.darkreading.com/vulnerabilities-threats/new-ai-challenges-test-ciso-teams-2025

We Need to Talk About the Insider Risk from AI

 

(Paul Martin CBE, Sarah Mercer – RUSI – 8 January 2025) As AI systems increasingly undertake tasks previously performed by humans, organisations face a growing security risk from artificial insiders as well as human ones. This complex security risk is poorly understood. Fortunately, some of the approaches that protective security practitioners use for managing risks from human insiders are also applicable to AIs. – https://www.rusi.org/explore-our-research/publications/commentary/we-need-talk-about-insider-risk-ai

Pall Mall Process to tackle commercial hacking proliferation raises more concerns than solutions

(Alexander Martin – The Record – 8 January 2025) A year on from the launch of the Pall Mall Process to tackle “the proliferation and irresponsible use” of commercial hacking tools, there are concerns among its participants that the initiative lacks the ability to actually change how these tools are traded and used. The market for what are formally called commercial cyber intrusion capabilities (CCICs) is growing, according to a consultation summary published by Pall Mall Process organizers Wednesday, which warned that the threats CCICs pose to national security and human rights “are expected to increase over the coming years.” – https://therecord.media/pall-mall-process-commercial-hacking-concerns

UK Government to Ban Creation of Explicit Deepfakes

(Phil Muncaster – Infosecurity Magazine – 7 January 2025) The British government has announced plans to criminalize the creation of sexually explicit deepfakes, with perpetrators facing up to two years behind bars if found guilty. It is already an offense to share or threaten to share intimate images, including deepfakes, following amendments to the Sexual Offences Act 2003 that were made by the Online Safety Act 2023. – https://www.infosecurity-magazine.com/news/uk-government-ban-creation/

Phishing Click Rates Triple in 2024

(James Coker – Infosecurity Magazine – 7 January 2025) The rate at which enterprise users clicked on phishing lures nearly trebled in 2024, according to new research by Netskope. More than eight out of every 1000 users clicked on a phishing link each month in 2024, up by 190% compared to 2023. – https://www.infosecurity-magazine.com/news/phishing-click-rates-triple/

From $22M in Ransom to +100M Stolen Records: 2025’s All-Star SaaS Threat Actors to Watch

(The Hacker News – 6 January 2025) In 2024, cyber threats targeting SaaS surged, with 7,000 password attacks blocked per second (just in Entra ID)—a 75% increase from last year—and phishing attempts up by 58%, causing $3.5 billion in losses (source: Microsoft Digital Defense Report 2024). SaaS attacks are increasing, with hackers often evading detection through legitimate usage patterns. The cyber threat arena saw standout players, unexpected underdogs, and relentless scorers leaving their mark on the SaaS security playing field. As we enter 2025, security teams must prioritize SaaS security risk assessments to uncover vulnerabilities, adopt SSPM tools for continuous monitoring, and proactively defend their systems. – https://thehackernews.com/2025/01/from-22m-in-ransom-to-100m-stolen.html

Atos, contractor for French military and intelligence agencies, dismisses ransomware attack claims

(Alexander Martin – The Record – 3 January 2025) Atos, the struggling French technology company that secures communications for France’s military and intelligence services, on Friday dismissed as “unfounded” a ransomware group’s claims to have compromised an internal company database. The extortion attempt comes as Atos, which employs around 90,000 people, is in negotiations to sell off its advanced computing division to the French State as the company attempts to restructure and avoid financial collapse. – https://therecord.media/atos-dismisses-ransomware-claims

New AI Jailbreak Method ‘Bad Likert Judge’ Boosts Attack Success Rates by Over 60%

(Ravie Lakshmanan – The Hacker News – 3 January 2025) Cybersecurity researchers have shed light on a new jailbreak technique that could be used to get past a large language model’s (LLM) safety guardrails and produce potentially harmful or malicious responses. The multi-turn (aka many-shot) attack strategy has been codenamed Bad Likert Judge by Palo Alto Networks Unit 42 researchers Yongzhe Huang, Yang Ji, Wenjun Hu, Jay Chen, Akshata Rao, and Danny Tsechansky. “The technique asks the target LLM to act as a judge scoring the harmfulness of a given response using the Likert scale, a rating scale measuring a respondent’s agreement or disagreement with a statement,” the Unit 42 team said. – https://thehackernews.com/2025/01/new-ai-jailbreak-method-bad-likert.html

Web3 Attacks Result in $2.3Bn in Cryptocurrency Losses

(James Coker – Infosecurity Magazine – 3 January 2025) Web3 security incidents resulted in over $2.3bn worth of cryptocurrency in losses in 2024, a 31.6% increase in the value stolen compared to 2023, according to new figures from blockchain security firm Certik. These losses took place across 760 incidents, 29 less than in 2023. The average amount stolen per hack was $3.1m in 2024, a 23% increase from 2023. – https://www.infosecurity-magazine.com/news/web3-attacks-cryptocurrency-losses/

6 AI-Related Security Trends to Watch in 2025

(Jai Vijayan – Dark Reading – 31 December 2024) Most industry analysts expect organizations will accelerate efforts to harness generative artificial intelligence (GenAI) and large language models (LLMs) in a variety of use cases over the next year. Typical examples include customer support, fraud detection, content creation, data analytics, knowledge management, and, increasingly, software development. A recent survey of 1,700 IT professionals conducted by Centient on behalf of OutSystems had 81% of respondents describing their organizations as currently using GenAI to assist with coding and software development. Nearly three-quarters (74%) plan on building 10 or more apps over the next 12 months using AI-powered development approaches. – https://www.darkreading.com/cyber-risk/6-ai-related-security-trends-watch-2025

Defense, Intelligence, and War

Ukrainian cyber market grows amid war but still lacks support and funding, report says

(Daryna Antoniuk – The Record – 8 January 2025) Ukraine’s cybersecurity market has grown rapidly in recent years, driven by rising threats, increased digitization and international assistance. However, new research shows that the sector remains heavily reliant on foreign aid and faces significant challenges, including insufficient investment, a lack of competitive advantage and weak regulations. The report by the Ukrainian consulting firm DataDriven tracked the value of the country’s cyber companies over the past eight years. Over that span the market increased fourfold, reaching an estimated value of $138 million last year. – https://therecord.media/ukraine-cybersecurity-market-study-datadriven

Frontiers

South Korea to Form Quantum Strategy Committee

(Matt Swayne – Quantum Insider – 9 January 2025) South Korea plans to establish a Quantum Strategy Committee by the first quarter of this year to drive its national quantum technology development, according to Vice Minister Lee Chang-yoon. The committee, created under the “Act on the Promotion of Quantum Science and Technology and Quantum Industry,” will involve government ministries, academia and industry to shape strategies for advancing quantum innovation. Despite delays due to political challenges, preparations for the committee’s launch are complete, with experts predicting 2025 will be a transformative year for South Korea’s quantum technology sector. – https://thequantuminsider.com/2025/01/09/south-korea-to-form-quantum-strategy-committee/

2025 Will See Huge Advances in Quantum Computing. So What is a Quantum Chip And How Does it Work?

(Muhammad Usman – Quantum Insider – 8 January 2025) In recent years, the field of quantum computing has been experiencing fast growth, with technological advances and large-scale investments regularly making the news. The United Nations has designated 2025 as the International Year of Quantum Science and Technology. The stakes are high – having quantum computers would mean access to tremendous data processing power compared to what we have today. They won’t replace your normal computer, but having this kind of awesome computing power will provide advances in medicine, chemistry, materials science and other fields. So it’s no surprise that quantum computing is rapidly becoming a global race, and private industry and governments around the world are rushing to build the world’s first full-scale quantum computer. To achieve this, first we need to have stable and scalable quantum processors, or chips. – https://thequantuminsider.com/2025/01/08/2025-will-see-huge-advances-in-quantum-computing-so-what-is-a-quantum-chip-and-how-does-it-work/

Quantum Sensing Technology Reveals Sub-Atomic Signals

(Matt Swayne – Quantum Insider – 7 January 2025) A Penn-led team of scientists have developed a quantum sensing method that detects signals from individual atoms, offering unprecedented precision in molecular analysis. The technique isolates single nuclei to reveal tiny differences in molecular structures, enabling advancements in fields like drug development and protein research. The discovery was made using nitrogen-vacancy centers in diamonds and combines decades-old theory with modern technology, opening new frontiers in quantum physics and spectroscopy. – https://thequantuminsider.com/2025/01/07/quantum-sensing-technology-reveals-sub-atomic-signals/

Alberta’s $70B Bet on AI Data Centers

(James Dargan – AI Insider – 7 January 2025) Alberta is positioning itself as a global hub for AI with a $70 billion project to build the world’s largest and most efficient data center, leveraging its abundant natural gas reserves. Premier Danielle Smith revealed the ambitious plan, citing Alberta’s vast energy resources — 200 trillion cubic feet of natural gas and advanced carbon capture infrastructure — as key advantages. – https://theaiinsider.tech/2025/01/07/albertas-70b-bet-on-ai-data-centers/

Microsoft to Invest $80B in AI-Driven Data Centers by FY 2025

(James Dargan – AI Insider – 6 January 2025) Microsoft has announced plans to invest $80 billion in fiscal year 2025 to expand its global network of AI-enabled data centers. The initiative aims to support the training of AI models and the deployment of AI and cloud-based applications. – https://theaiinsider.tech/2025/01/06/microsoft-to-invest-80b-in-ai-driven-data-centers-by-fy-2025/

Against the Corporate Capture of Human Connection

(Susie Alegre – Centre for International Governance Innovation – 30 December 2024) The dream of a synthetic idealized partner to avoid the mess of dealing with a real person has captured the human imagination for millennia. In the ancient Greek myth, the king Pygmalion found his happily ever after with the statue Galatea that the goddess Aphrodite obligingly breathed life into so that he could have children without engaging with an actual woman’s form, which he found repulsive. The 2013 film Her saw Joaquin Phoenix falling for his chatbot assistant, Samantha, in a clear reflection that these myths live on. In the era of generative and emotional artificial intelligence (AI), such phenomena are no longer the stuff of myths or science fiction. Anyone can have their own synthetic, personalized AI relationship in their pocket. But a lawsuit filed this month about the potential manipulative powers of AI chatbots, and their impact on children in particular, is the latest evidence that we should think carefully about the impact of emotional AI on human interactions. – https://www.cigionline.org/articles/against-the-corporate-capture-of-human-connection/