Navigating AI in Social Care: Practical Guidance for Providers
Katie is a leading voice in AI and digital transformation in adult social care, known for her focus on the ethical and responsible use of AI. Through her work with organisations such as Oxford University, Digital Care Hub, and NHS England, she is helping to ensure AI supports care professionals, reduces administrative burden, and strengthens human-centred care. She plays a key role in shaping national discussions and is at the forefront of ensuring digital change benefits both staff and those receiving care.
Daniel Casson is one of the UK’s leading experts on digital transformation in social care, bringing practical, real-world insight into how AI is adopted across the sector. He works closely with care providers, policymakers, and technology organisations, with expertise spanning ethics, workforce impact, care quality, and innovation. His work focuses on helping organisations implement AI in a responsible, effective, and human-centred way.
Together, Katie and Daniel, alongside Professor Caroline Green from the Institute of Ethics and AI at Oxford University, have been exploring what responsible AI use means for social care, with a strong focus on the experiences of people receiving care and frontline staff. Drawing on lessons from the past decade of digital transformation, they aim to ensure the sector avoids repeating previous mistakes. Their shared message is that AI adoption should be sector-led, inclusive, and aligned with the core values of care, supporting rather than replacing the human element.
Summary of the meeting and the key takeaways that providers can use to get ready for using AI in Social Care
Daniel and Katie were given 6 questions in advance of this meeting for them to prepare answers to.
Q1. What decisions should providers be making in the next 12 months to avoid falling behind?
- Providers in the next 12 months should focus on becoming AI ready by actively engaging with staff, people they support, and families to understand what AI is being used and how, including identifying existing “Shadow AI” use. They should build a roadmap and develop a basic AI policy outlining acceptable tools and uses, even if it evolves over time.
- Organisations must communicate clearly with staff, recognising many are already using tools like ChatGPT, and ensure safe, informed use rather than making assumptions about knowledge. Alongside this, providers should define their risk appetite, carry out risk assessments, and begin trialling AI in lower-risk areas while consulting widely across the organisation.
- AI should be approached as a way to enhance human care, improving back-office efficiency and making better use of data to support care planning. Providers need to be clear on what they want AI to achieve (and what they don’t), taking a tailored approach based on their organisation’s needs while investing in strong foundations. Think widely on the scope for AI use and focus on strategic goals.
- Leaders are encouraged to explore AI tools themselves (safely, without personal data) to build understanding, avoid misinformation, and support more informed conversations, helping bridge the trust gap between professionals and the public.
- Finally, providers should be aware of the wider impacts, including increased scrutiny from families using AI for information, AI-generated complaints, and growing expectations, highlighting the need for preparedness, transparency, and ongoing dialogue across social care.
Q2. What’s the biggest mistakes providers are currently making with AI?
- The biggest mistake providers could make is to use AI for everything without a clear goal or training, treating it as a universal solution rather than asking whether it is the right tool to achieve a specific outcome. This would lead to poor value, as seen in examples where AI tools were introduced without direction or expectations.
- Providers should encourage staff involvement and open communication, allowing those with stronger AI skills to share insights, raise concerns, and contribute solutions, which can improve oversight and implementation.
- A critical issue is the lack of governance and structure, some organisations have even received poor CQC ratings due to weak governance around AI, while others have achieved positive outcomes where it is well managed. Equally important is transparency, ensuring families and stakeholders are informed when AI is being used and how decisions are being made.
- Mistakes also arise from poor procurement practices, where organisations fail to follow ICO guidance, and treat AI as a technology project rather than a human-focused one.
- Providers should pilot and evaluate AI tools properly, measuring their impact against clear baselines, and consider safer approaches such as closed-loop systems to protect sensitive data. They must also avoid being over-reliant on AI, remaining critical of outputs (e.g. in HR processes) and recognising its limitations.
- Ultimately, success comes from asking the right questions early, being intentional in approach, and pre-empting mistakes rather than reacting to them.
Q3. What skills will providers need to start building in their teams over the next 2-3 years and where can they find support?
- Over the next 2–3 years, providers need to build strong digital skills across the workforce, recognising that while digitisation has rapidly increased (e.g. digital care records and rostering), parts of the workforce, particularly those most at risk of digital exclusion, need additional support and training.
- A key priority is developing baseline digital competencies, supported by resources like the Skills for Care digital skills framework, which outlines both essential skills and more advanced capabilities for leadership and procurement roles. While AI-specific training is still limited, providers can access free resources from organisations such as the Department for Science, Innovation and Technology, Digital Care Hub, AI Care Alliance, and the Oxford Project.
- Providers should focus on building AI confidence, encouraging staff to safely experiment with tools, understand the different ways of using them, and adapt them to their roles rather than relying on a single approach. This includes promoting data protection awareness, ensuring that staff understand the risks of sharing sensitive information. Digital Care Hub have free data protection e-learning.
- There is also a growing need for internal leadership and governance capability, with some organisations establishing AI ethics groups or dedicated AI leads to guide responsible use and decision-making.
- Ultimately, providers must support staff to engage critically with AI, treat it as a tool to enhance work (not replace judgement), and carefully assess how AI features introduced by suppliers fit their needs. Building these skills now will help organisations keep pace with change while ensuring safe, effective, and confident use of AI.
Q4. If you could just talk about your experience of ethics in AI and what ethical lines do you think providers need to be very clear about not crossing?
- Providers must prioritise transparency (“truth”) in how they use AI, being clear with people, families, and staff about where and how it is used in decision-making. AI should always remain human-led, ensuring empathy, professional judgement, and human oversight are central, especially for decisions that impact people’s care, health, or employment.
- A clear ethical line is to avoid fully automated decision-making in social care. Final decisions must always involve humans, as automation risks undermining the values of care and could negatively affect both staff and those supported.
- Providers must also actively address bias and discrimination in AI systems, recognising that AI can amplify existing inequalities present in data. This requires ongoing review, testing, and staff awareness to identify and challenge biased outputs.
- Following established CQC and ICO guidance and ethical frameworks will help organisations stay within safe boundaries. However, responsibility also extends to tech suppliers. Care providers need to question and understand the ethics behind the technology they procure, ensuring it aligns with standards across the supply chain.
- Strong collaboration is essential: working with staff, suppliers, and the wider sector to promote a responsible, ethical approach, amplifying the voices of vulnerable people, and continuously improving the systems (including challenging and feeding back on biased or problematic outputs).
- Overall, ethical AI in social care requires openness, accountability, active risk management, and keeping human judgement at the heart of all decisions.
Q5. What questions should providers be asking vendors before they buy anything? Are there red flags that they should be looking for?
- When selecting AI vendors, providers should assess alignment with recognised frameworks such as the AI and Care Alliance Tech Pledge, focusing on transparency, responsible development, and evidence of real-world impact and outcomes. Vendors should clearly explain how their AI models work, how they are updated, and how they support continuous learning and improvement.
- Providers should expect greater accountability from suppliers, with a shift toward proactive transparency rather than reactive reassurance. This includes understanding how tools are built, tested, and monitored over time.
- Key questions should cover regulation and safety, such as whether the supplier aligns with medical device standards, maintains audit trails, and has appropriate governance (e.g. clinical safety oversight). These indicators show a serious, structured approach to risk and quality.
- Providers should also consider interoperability (how systems integrate), though this is expected to improve with upcoming guidance. Increasingly, some care organisations are even exploring building their own AI tools, reflecting growing capability in the sector.
- A strong positive sign is “radical transparency”, vendors openly sharing how their systems work, like a “greenhouse” model where processes are visible and open to scrutiny. Conversely, red flags include lack of clarity, weak governance, or reluctance to explain how systems operate.
- Ultimately, providers should ask clear, practical questions, ensure suppliers meet ethical and regulatory expectations, and prioritise partners who demonstrate trustworthiness, openness, and responsibility in how their AI is developed and used.
Q6. Have you got any ideas about what’s going to change or what’s going to come in in terms of regulation over the next few years?
- Over the next few years, regulation around AI in social care is likely to evolve through guidance rather than strict new rules, with bodies like the CQC clarifying expectations but not acting as direct technology regulators. Instead, providers will need to align with existing frameworks, including the Health and Social Care Act, while monitoring guidance from multiple bodies.
- Where AI is classified as a medical device, regulation is clearer under organisations like the MHRA, although current approaches still rely heavily on self-assessment. Wider developments are expected from the AI Commission, but social care is currently out of scope, meaning gaps in clarity may remain.
- A key challenge is the lack of consistency across health and social care, with different expectations from the NHS, local authorities, and care providers. This may lead to variation in interpretation at a local level, meaning providers must engage directly with commissioners and regulators to understand expectations around transparency, consent, and implementation.
- There is likely to be a gradual move toward greater alignment between health and social care standards, especially as integrated systems develop, alongside increasing expectations to follow emerging best practice guidance.
- Another important development is the need for stronger evidence and evaluation of AI tools. Unlike established areas with long-term data (e.g. radiography), AI in social care currently lacks a robust evidence base, which affects trust and adoption. Building this evidence will be key to future regulation and acceptance.
- Overall, the next few years will involve refining existing regulations, increasing consistency, building evidence, and navigating differing perspectives within the workforce, as organisations and society continue to shape their approach to AI.
Please see below links that were given at the meeting of references for further reading/training and research that has been conducted in this area
Artificial intelligence in health and social care: CQC’s role, expectations and plans
The AI in Social Care Alliance – vision, principles and priority actions
https://www.digitalcarehub.co.uk/ai-and-robotics/ai-in-social-care-alliance/
One in seven people have used AI instead of seeing a GP, study finds
https://www.kcl.ac.uk/news/one-in-seven-people-have-used-ai-instead-of-seeing-a-gp-study-finds
How physicians are using LLMSs
https://www.jmir.org/2025/1/e76941?utm_source=copilot.com
Data security Protection Toolkit
https://www.digitalcarehub.co.uk/digital-skills-and-training/elearning/
Top tips for care providers
A range of digital products, tools, advice and resources to support people working in adult social care with their digital skills and knowledge.
https://beta.digitisingsocialcare.co.uk/develop-digital-skills
Responsible use of Generative AI in social care: Guidance
Upskilling workers with FREE AI skills and confidence to boost productivity at work
https://aiskillshub.org.uk/aiskillsboost/
Good Things Foundation’s AI Gateway, where you can learn about artificial intelligence
https://learning.goodthingsfoundation.org/ai
Pledge by tech suppliers on the responsible use of AI in social care
Ctrl + Care is your weekly download of the biggest stories in adult social care, tech, and cyber security, all in under 30 minutes.