What should the law do about deepfakes?

From Taylor Swift to the Royal Family – deepfakes are rarely out of the news. BDFI’s Prof. Colin Gavaghan asks what we can do to protect ourselves and if lawmakers should be doing more. 

Credit: Kenzie Saunders/flickr

The camera does lie. It always has. For as long as we’ve had photography, we’ve had trick photography. Some of this is harmless fun. I remember as a child delighting in forced perspective photos that made it look like I was holding a tiny building or relative in the palm of my hand. Some of it is much less than harmless. Stalin was notorious for doctoring old photographs to excise those who had fallen from his favour.

The development of AI deepfakes has taken this to a new level. It’s not just static images that can be manipulated now. People can be depicted saying and doing things that are entirely invented.

Credit: GabboT/flickr

If anyone hadn’t heard of deepfakes before, the first few months of 2024 have surely remedied that. First, in January, deepfake sexual images of Taylor Swift – probably the world’s most famous pop star – were circulated on X and 4chan. This month, deepfakes were back among the headlines, when rumours circulated that a family picture by the Princess of Wales had been digitally altered by AI.

In some ways, the stories couldn’t be more different. The Taylor Swift images were made and circulated by unknown actors, without the subject’s consent, and in a manner surely known or intended to cause embarrassment and distress.

Source: The Guardian

Princess Kate’s picture, in contrast – which it turns out was more likely edited by more basic software like Photoshop – was made and shared by the subject herself, and any embarrassment will be trivial and to do with her amateur photo editing skills.

In other ways, though, the two stories show two sides of the challenge these technologies will pose.

The challenges posed by intimate deepfakes are the more obvious, and have been known about long before Taylor Swift became their most high profile victim. As with ‘revenge porn”, the victims are overwhelmingly women and girls, and the harm it can do is well documented.  

There have been legal responses to this. The new Online Safety Act introduced a series of criminal offences aimed at the intentional sharing of “a photograph or film which shows, or appears to show, another person in an intimate state” without their consent. The wording is specifically intended to capture AI generated or altered images. These offences are not messing around either. The most serious of them carries a maximum prison sentence of two years.

Source: X

That sort of regulatory response targets the users of deepfake technologies. Though it’s hoped they have some deterrent effect, they are retrospective responses, handing out punishment after the harm is done. They also don’t have anything to say about a potentially even more pernicious use of deepfakes; the generation of fake political content. In 2022 a fake video circulated of Ukrainian president Volodymyr Zelensky appearing to announce the country’s surrender to Russia. And in January this year, voters in new Hampshire received a phone call from a deepfake “Joe Biden”, telling them not to vote in the Democrat primary.

Unlike intimate deepfakes, political deepfakes don’t always have an obvious individual victim. The harms are likely to be more collective – to the democratic process, perhaps, or national security. It would be possible to create specific offences to cover these situations too. Indeed, the US Federal Communications Commission acted promptly after the Biden deepfake to do precisely that.

An alternative response, though, would be to target the technologies themselves. The EU has gone some way in this direction. Article 52 of the forthcoming AI Act  requires that AI systems that generate synthetic content must be developed and used in such a way that their outputs are detectable as artificially generated or manipulated. The Act doesn’t specify how this would be done, but suggestions have included some sort of indelible watermark.

Will these responses help? It’s likely that the new offences will deter some people, but as with previous attempts to regulate the internet, problems are likely to exist with identification – you can’t punish someone for creating such images if you can’t find out who they are – and with jurisdiction.

What about the labelling requirements? There are technical doubts about how easy it will be to circumvent the detection system. And even when content is labelled as fake, it’s uncertain how this will affect the viewer. Early research suggests we should be cautious about assuming warnings will insulate us against fakery, with some researchers pointing out a tendency to overlook or filter out the warning: “Even when they’re there, audience members’ eyes—now trained on rapid-fire visual input—seem to unsee watermarks and disclosures.”

As for intimate deepfakes, detection systems may help a bit. But I’m struck by how the harm to these women and girls seems to persist, even when the images are exposed as fakes. In a case in Spain last year, teenaged girls had deepfake nudes created and circulated by teenaged boys. As one of the girls’ mothers told the media, “They felt bad and were afraid to tell and be blamed for it.” This internalisation of blame and shame by the victims of these actions suggests that a deeper problem may lie in persistent and damaging attitudes towards female bodies and sexuality, rather than any particular technology.

Source: bandeepfakes.org

Maybe in a better future, intimate deepfakes won’t cause that level of harm. We might hope that schoolmates and neighbours will rally round the victims, and that any stigma will be reserved for the bullies and predators who have created the images. We can hope. But meanwhile, these technologies are being used to inflict considerable suffering. One solution that is gaining support would be to ban deepfake technologies altogether. Maybe the potential for harm just outweighs any potential benefit. That was certainly the view of my IT Law class last week!

But what precisely would be subject to the ban? That question brings me back to Kate’s family pic. If we are to ban “deepfakes”, where would we draw the line? Does image manipulation immediately become pernicious when AI is involved, but remain innocent when it’s done with established techniques like Photoshop? If lawmakers are going to go after the technology, rather than the use, then we’re going to have to think about precisely what technology we have in our sights.

‘If you can’t tell, does it matter?’ Do we need new law for human-like AI?

With the persistent rise in chatbots and other human-like AI, Prof. Colin Gavaghan, BDFI’s resident tech lawyer, asks: do we need regulatory protection from manipulation?

Stills from WestWorld filmRobots and AI that look and act like humans is a standard trope in science fiction. Recent films and tv series have supplemented the shelves of books taking this conceit as a central concept. One of the most celebrated – at least in its first season – was HBO’s reimagining of Michael Crichton’s 1973 film WestWorld.

The premise of WestWorld is well known. In a futuristic theme park, human guests can pay exorbitant sums to interact with highly realistic robots or ‘hosts’. In an early episode, a human guest, William, is greeted by Angela, a “host.” When William enquires as to whether she is “real” or a robot, Angela responds: ‘Well if you can’t tell, does it matter?’

As we move through an era where AI and robotics acquires ever greater realism in its representations of humanity, this question is acquiring increasing salience. If we can’t tell, does it matter? Evidently, quite a lot of people think it matters quite a lot. For instance, take a look at this recent blog post from the excellent Andres Gaudamuz (Technollama).

But why might it matter? In what contexts? And what, if anything, should the law have to say about it?

What’s the worry about humanlike AI?

Writing in The Atlantic a few months ago, philosopher Dan Dennett wrote this:

“Today, for the first time in history, thanks to artificial intelligence, it is possible for anybody to make counterfeit people who can pass for real in many of the new digital environments we have created. These counterfeit people are the most dangerous artifacts in human history, capable of destroying not just economies but human freedom itself.”

The most dangerous artifacts in human history?! In a year when the Oppenheimer film – to say nothing of events in Ukraine – have turned our attention back to the dangers of nuclear war, that is quite a claim! If we are to make sense of Dennett’s claim, far less decide whether we agree with it, we need to understand what Dennett means by “counterfeit people”. The term could refer to a number of things.

One obvious way in which AI can impersonate humans is through applications like ChatGPT, that can generate text indistinguishable from that generated by humans. When this is linked to a real-time conversational agent – a chatbot or an AI assistant – it can result in a conversation in which the human participant might reasonably believe the other party is also a human. Google’s “Duplex” personal assistant added a realistic spoken dimension to this in 2018, its naturalistic “ums” and “ahs” giving the impression of speaking to a real PA.

More recently, the Financial Times reported that Meta intends to release a range of AI “persona” chatbots, including one that talks like Abraham Lincoln, to keep users engaged with Facebook. Presumably, users will be aware that these are chatbots (does anyone think Abe Lincoln is actually on Facebook?) In other cases, the true identities of the chatbots will be concealed, as when bot accounts are used to spread propaganda and disinformation.

Those examples read and sound like they might be human. But AI can go further. Earlier this year, Sen. Richard Blumenthal (D-CT) kicked off a Senate panel hearing with a fake recording of his own voice, in which he described the potential risks of AI technology. So as well as impersonating humans, we now have to be alert for AI impersonating particular humans.

Soul MachinesAs the technology evolves, we’ll find AI that can impersonate humans across a whole range of measures – not only reading and sounding human, but looking and acting like it too. This is the sort of work being done by Soul Machines, whose mission is to use “cutting edge AI technology … to create the world’s most alive Digital People.”

Other than a vague unease caused by these uncanny valley denizens, why should this bother us?

One of the main concerns relates to manipulation. Writing in The Economist in April, Yuval Noah Harari claimed that AI has “hacked the operating system of human civilisation”. His concern was with the capacity of AI agents to form faux intimate relationships, and thereby exert influence on us to buy or vote in particular ways.

This concern is far from fanciful. Research is already emerging, suggesting that we are, if anything, more likely to trust AI-generated faces. Imagine an AI sales bot that is optimized to look trustworthy, and combine that with software that lets it appear patient and friendly, but also able to read our voices and faces so it knows exactly when to push and when to back off.

So great are these concerns that we have already seen some legal responses. In 2018, California introduced the BOT (Bolstering Online Transparency) Act, which bans the use of pretend-human bots if they’re used to try to influence purchasing or voting decisions. Art 52 of the EU’s new AI Act adopts a similar measure to the Californian one.

Are mandatory disclosure laws the answer?

AI agents are certainly being optimized to pass for human, with a view to sell, persuade, seduce and nudge us into parting with our attention, our money, our data, our votes. What’s less obvious is how much mandatory disclosure will insulate us against that. Will knowing that we’re interacting with an AI protect us against its superhuman persuasive power?

There is some reason to think it might play a role. One study from 2019 found that potential customers receiving a cold call about a loan renewal offer were as or more likely to take up the offer when it was made by an AI. But this advantage largely dissipated when they were told in advance that the call was from a chatbot.

Interestingly, the authors of the 2019 paper reported that late disclosure of the chatbot’s identity – that is, after the offer has been explained, but before the customer makes up their mind about whether to accept it – seemed to cancel out the antipathy to chatbots. This leads them to the provisional conclusion that experience of talking to chatbot will allay some of their concerns about it. In other words, as we get more used to talking with AIs, our intuitive suspicion of them will likely dissipate.

Another reason to be somewhat sceptical of mandatory disclosure solutions is that telling me whether something was generated by AI tells me little or nothing about whether it’s true, or about whether the person I’m talking to is who they claim to be. Ultimately, I don’t really care if content comes from a bot, a human scammer, a Putin propaganda troll farm, or a genuine conspiracy theorist. Is “Patrick Doan”, the “author” of the email I received recently, a person or a bot? Who cares. He/it is clearly phishing me either way:

Phishing email

So much for cognitive misrepresentation. What about emotional manipulation? Will knowing that I’m talking to an AI help us resist the sort of emotional investment that will help the AI lead me into bad decisions?DuoLingo owl

My answer for now is: I just don’t know. What I do know, from many hours of personal experience, is that I am by no means immune to emotional investment even in the very weak AI we have now. They don’t even need to look remotely human.  I’m even a sucker for the blatant emotional nudges from the little green owl if I don’t do my DuoLingo practice!

Vulnerable and lonely people are going to be even easier prey. Phishing and catfishing are likely still to be problems, whether the fisher is a human or an AI. Imagine trying to resist that AI Abraham Lincoln (or Taylor Swift or Ryan Gosling), when it’s been optimized to hit all the right sweet-talking notes.

Targeted steps forward

If this all sounds like a counsel of despair, it isn’t meant to. I think there are meaningful steps that can be taken to mitigate the manipulative threat posed by human-like AI. But I suspect those measures will likely have to be properly targeted if they’re to have that effect. Simply telling me that I’m talking to a “counterfeit person” is unlikely to be enough to protect me from its persuasive superpowers.

We could, for instance, consider seriously the prospect of going hard after this sort of technology, or the worst examples of it anyway. Under the EU AI Act, those AI systems which are deemed to present an unacceptable risk are to be banned outright. This includes AI that deploys subliminal techniques beyond a person’s consciousness in order to materially distort a person’s behaviour in a manner that causes or is likely to cause that person or another person physical or psychological harm.

Perhaps there will soon be a case for adding highly persuasive AI systems to that list.

The UK Government seems to be going in a very different direction with regard to AI regulation, and the protections of the AI Act are unlikely to apply here. But other options exist. We could, for instance, consider stronger consumer law protections against manipulative AI technologies, to match those we have for “deceptive” and “aggressive” sales techniques.

In truth, I don’t have a clear idea right now about the best regulatory strategy. But it’s a subject I’m planning to look into more closely. Maybe it does matter if we can tell AI from human – at least to some people, at least some of the time. But on its own, I fear that knowledge will be nowhere near enough to prevent ever smarter AI, to use Harari’s words, hacking our operating systems.

This content is based on a paper given at the Gikii 2023 Conference in Utrecht, and at this year’s annual guest lecture at Southampton Law School. Colin is grateful for the helpful comments received at both. 

Network applications as an enabler for AI-driven autonomous networking

BDFI academic Dr Xenofon Vasilakos recently attended the IEEE ICC 2023 Industry Forum and Exhibition in Rome where he gave a speech to the IF&E workshop. In this blog he goes into detail about the topics covered in the speech, as we move from the fifth (5G) towards to the sixth (6G) generation of telecommunication networks.

5GASP explores self-managing and self-organizing automation for the development of sixth generation (6G) intelligent future networks. This is achieved through an ecosystem of specialized AI-driven network applications that enable automation. These applications fulfil the automation requirements of other “enhanced” network applications or services. The prototypes of these applications include network and performance prediction systems that enable proactive resource management and a human-centric approach, adapting to the dynamic nature of 6G networks and users without the need for human intervention. This AI-based automation provides improved network and service quality, while also ensuring compliance with business requirements and enhancing service agility.

Below, we provide a summary of the prototypes for AI-driven enablers, self-organised, or managed network applications.

(1) Efficient MEC Handover (EMHO) network application (Univ. of Bristol, AI-driven Autonomy enabler)

The functioning of this network application depends on collaborative machine learning (ML) predictions to maintain and potentially improve the quality of service provided by enhanced network applications operating on a multi-access edge computing (MEC) platform. The existing prototype utilizes mobile radio resource control (RRC) monitoring data along with an additional ML layer consisting of cooperative models that predict MEC handovers.

(2) Virtual On-Board Unit (vOBU) provisioning Network Application (OdinS, AI self-organisation)

This network application deploys a digital twin (DT) of a car on-board unit (OBU) on the nearest MEC node of its location. The DT can be “migrated” to car’s nearest edge as a twin (virtual) vOBU acting as a proxy, and its migration automatically begins upon cars’ movement. To avoid bottlenecks, this network application can pose an intent for forecasting future car locations with EMHO’s mobility prediction ML, thus allowing it to proactively deploy vOBU.

(3) PrivacyAnalyser Network Application (Lamda Networks, self-management)

PrivacyAnalyser is a cross-vertical cloud-native application running either at network Core or MEC. Among other features, it caters for ML network data classification from UE and/or IoT devices, and privacy evaluation and analysis. Also, PrivacyAnalyser is converging toward ML-based network management and orchestration via EMHO’s exposed ML predictions, enabling smart scale-in/out MEC pods proactively, better than the default container autoscaling for improving energy efficiency.

(4) Remote Human Driving Network Application (DriveU.auto, AI-driven self-management & self-organisation)

This Network Application enables remote autonomous vehicle operation in unusual/dangerous situations. The intent is to ensure reliable, low-latency, high-quality real-time video transmission via AI-optimised network latency, but also via EMHO Network Application handover predictions to automatically deploy appropriate applications with optimised slice features matching dynamic needs.

Future Steps, Impact & sociotechnical aspects

5GASP aims to establish an Open Source Software (OSS) repository and a VNF marketplace that caters to small and medium-sized enterprises (SMEs). It also focuses on fostering a community of network application developers by providing them with tools and services. These resources enable developers to achieve the following goals: (i) implement AI-driven network automation in network applications to improve network quality with minimal human intervention by capturing business and other intents through continuous monitoring, (ii) validate and certify network services early on to ensure alignment with business and other sociotechnical goals, and (iii) prioritize inter-domain use-cases for daily testing, validation, and ensuring security and trust of third-party intellectual property rights (IPR) in their testbeds.

The key lessons learned so far can be summarized as follows:

  • AI-driven automation plays a vital role in enhancing network and service automation by minimizing the need for human intervention and improving quality of service (QoS). Moreover, it allows the adoption of higher-level policies through proper orchestration decisions. Therefore, several sociotechnical aspects can be captured by translating key value indicators (KVIs) to network performance KPIs targets for AI enabler applications.
  • AI-driven network applications and the consumption of AI-driven artefacts (such as predictions or dynamic network orchestration suggestions) make 6G network automation achievable. Again, this can enable the adoption/imposition of sociotechnical targets and policies.

As for the next steps, the project has achieved a level of maturity where network applications are already deployed using the developed tools and procedures. The project is currently seeking network application developers, individuals or SMEs, outside of the consortium who are interested in validating their 5G applications and adopting the 5GASP methodology, tools, and innovative 6G automation network applications.

Related work

[1] A. Bonea et. al, Automated onboarding, testing and validation for Network Applications and Verticals, ISSCS Iasi, 2021.

[2] Kostis Trantzas et al., An automated CI/CD process for testing and deployment of Network Applications over 5G infrastructure, IEEE International Mediterranean Conference on Communications and Networking, 7–10 September 2021.

[3] X. Vasilakos et al., Towards Low-latent & Load-balanced VNF Placement with Hierarchical Reinforcement Learning, IEEE International Mediterranean Conference on Communications and Networking, 7–10 September 2021.

[4] M. Bunyakitanon et al., HELICON: Orchestrating low-latent & load-balanced Virtual Network Functions, IEEE ICC 2022.

[5] V. A. Siris et al. Exploiting mobility prediction for mobility & popularity caching and DASH adaptation, IEEE 17th International Symposium on A World of Wireless, Mobile and Multimedia Networks, 2016.

[6] R. Direito, et al., Towards a Fully Automated System for Testing and Validating Network Applications, NetSoft 2022, 2022.

[7] X. Vasilakos et al., Towards an intelligent 6G architecture: the case of jointly Optimised handover and Orchestration, WWRF47, 2022.

[8] N. Uniyal et al., On the design of a native Zero-touch 6G architecture, WWRF47, 2022.

 

Connected communities: are hybrid futures the way forward?

Following the publication of the ‘Post’ Pandemic Hybrid Futures report, Ella Chedburn from Knowle West Media Centre reflects on the pro and cons of connecting remotely during Covid, and what positives we should be taking forward from our different experiences of connecting during the pandemic.

Knowle West Fest

For many of us, the Covid-19 pandemic involved a huge shift from in-person to digital encounters across all areas of life. Here at KWMC, from the very first lockdown we knew we needed to find ways to keep working with and stay connected to our community, so we got creative with digital and blended ways of working. There were many positives to connecting remotely, through online platforms / posted packs etc. For some people, joining meetings, events or workshops from home was suddenly possible and more accessible. However, there were lots of negatives to purely online spaces too – not everyone has access to webcams or is familiar using technology, and some of these spaces had negative health impacts too. 

As we emerged from lockdowns, we wondered: could we get the best of both worlds by merging online and physical (‘hybrid’) spaces? We explored this in our ‘Come Together’ programme in 2021 and learned so much about the vices and virtues of these hybrid setups. We have lots of useful resources and examples on the website for anyone to use. However, as 2022 rolled around it became more and more tempting for institutions to forget these learnings and revert to in-person events that are often easier to run. 

The ‘Post’ Pandemic Hybrid Futures project came at the perfect time for us to pause and reflect on what learnings we could realistically carry forward from the pandemic. Through this collaboration, we were able to further develop some of the hybrid tools and methods we had learnt from workshops, community events, live broadcasts, festivals and blended programmes. We focused our collaboration on a specific experiment – how could we make a local community festival (Knowle West Fest) more accessible through hybrid means? 

Learning from the process

From the Knowle West Fest (KWfest) experiments one of our main learnings was that a rough-and-ready style works really well when it comes to livestreams. It seemed that the more authentic and casual style of Facebook Live resonated with many of our audiences. People in the physical space were also much more relaxed about being featured in a Facebook Live, with many seeming excited to talk on camera. Plus, the more informal nature meant that any pauses from lack of internet felt far less painful in both the online space and the physical space compared to Zoom. This livestream was also not too taxing on our staff, so it is realistic for us to continue doing them long-term. The biggest surprise was the success of our Facebook livestream afterwards – gaining over 1,000 views during the following week. Here we learned the importance of allowing digital audiences to engage in their own time.  

In comparison, only a couple of people joined our Zoom livestream. While marketing it, a few people responded negatively to the idea of Zoom – associating it with work and lockdown. People also expect events on this platform to be more professional and smoothly run, which adds pressure to staff. Despite our best efforts to market the space as a ‘cozy online portal’, these workplace associations will take more effort to overcome. Instead, we recommend using Zoom to fully engage in a single activity, allowing participants to get hands-on and make the most of the more personal space. Or even creating a pre-recorded complimentary offering to access from home instead. These have both worked very well in our previous projects. 

PostcardsAlongside our two livestream experiments, we left postcards around the festival for people to send to friends and family via a ‘post box’ in the cafe. On the back of the postcards was a link to a YouTube playlist of acts playing at the festival. Surprisingly, this activity went down particularly well with children and has a lot of scope for further experimentation such as adding art, or posting to (consenting!) strangers, or posting back and forth between people. It can also be less intense for staff to run and eliminates the stress of technology failures. After the festival we sent out craft packs to some people with links to online content – again demonstrating that to access a festival experience it doesn’t all have to synchronise or be live. 

The BDFI partnership 

BDFI’s aim to create more inclusive, sustainable and prosperous digital futures aligned well with our ethos at KWMC.  

BDFI’s support was invaluable in helping us to collate all our previous research and reflect on it from both internal and external perspectives. This allowed us to fully absorb and integrate our learnings then use them as a springboard for more experimentation.  

On a practical level, the extra staff from BDFI meant that we had enough people power to confidently deliver the hybrid elements. We learned the hard way through the Come Together project that hybrid events often need double the staff and can be more demanding for facilitators and producers, so it is important that they are properly resourced and well planned.  

Next steps

At KWMC, we hope to cultivate a more inclusive future by combining the best of digital and physical spaces. We are also keen to ensure that Knowle West communities continue to benefit from the research and experiments that they have participated in. We will be sharing these learnings with the 2023 KWfest producing team and exploring ways in which we can share the research more broadly with those working in the education, community, creative and charity sectors. 

Do Pixels Have Feelings Too?

BDFI co-director Professor Daniel Neyland hosted a fascinating and informative lecture about the ethics around artificial intelligence. Here he follows up that lecture with a thought-piece on the proliferation of AI, ethical principles and questions that can be applied, and the importance of trust and truth.

Daniel Neyland lecture

We appear to be moving into a period where the number of AI applications being launched is proliferating rapidly. All indications are that these applications will utilize a range of data, and operate at a speed and on a scale that is unprecedented. The ethical impact of these technologies – on our daily lives, our workplaces, modes of travel and our health – is likely to be huge.

This is a familiar story – we have perhaps heard similar narratives on previous occasions (for example in relation to CCTV in the 1990s, the internet in the late 1990s and early 2000s, biometric IDs from the early 2000s until around 2010, smartphones from around 2008 onwards, and so on). We are always told as part of these narratives that trying to address the impact emerging through these technologies will be incredibly difficult. However, the development of AI systems does seem to pose further specific challenges.

Firstly, for the most part, AI developments are even more opaque than some of the other technologies we have seen developed in recent decades. We don’t get to see the impacts of these systems until they are launched into the world, we may not even be aware that such systems exist before they are launched. In order to assess the likely problems specific AI applications will create, we need to open up the design and development stage of these systems to greater scrutiny. If we can intervene at the design stage, we might have a greater chance of reducing the number and range of harms that these systems might otherwise create.

Secondly, with generative AI and machine learning neural networks, systems have a certain amount of autonomy to produce their outputs. This means that if we want to manage the ethics of AI, we cannot work with the designers and developers of these systems alone. We need to work with the AI. Key to success here will be to engage with carefully bounded experiments to assess how AI engages with the social world, in order to assess its likely impacts and any changes to system design that are needed. We have an imperative to experiment with AI before it is launched into the world, but this imperative is in danger of being swept aside by the current drive to gain a market advantage by being the first mover in any particular AI application.

Thirdly, when we do have access to these AI applications, we need to attune our ethical assessment to the specific technology in focus. Not all AI is the same. In this lecture, I provide a range of broad ethical principles that draws on existing work in the field, but I also demonstrate how these principles can be given a specific focus when looking at a particular AI application – a machine learning, neural network that uses digital video to do emotion recognition.

I utilize broad ethical principles to raise questions regarding how a specific AI system can be re-designed. The ethical principles and associated questions set out one way we can discover and address concerns in the development of new AI systems. These include:

  • Consultation – at the design stage, how can we actively foster engagement with emerging AI systems to assess perceptions, trust and sentiment, for example, toward an emerging system?
  • Confidence – do we have confidence that the system will perform as we expect, how can we assess confidence (what kinds of experiments might we carry out, for example, to test how well a system works), and how can we address concerns raised by a system that is not operating as anticipated?
  • Context – in what setting is the system designed for use and what concerns arise from switching contexts of use?
  • Consequence – what happens as a result of the system being used, who is subject to AI decision making and for what purpose?
  • Consent – how can people give agreement that they should be subjects of AI decision making, that their data should be processed by AI, or that they are happy to work with an AI system in their workplace?
  • Compliance – what are the relevant legal and regulatory frameworks with which a system must comply? How might we design regulatory compliance into the technology?
  • Creep – if we carry out an ethical assessment in relation to a new and emerging technology in one use case, how might we guard against or assess the issues that might arise if that technology is used in other contexts?

These ethical principles and questions are not designed to be exhaustive, but I suggest, these need to be applied, developed, added to or moved in different directions when they are applied to specific technologies under development. They seem to represent a useful starting point for asking questions. In the lecture on neural networks for machine learning, I suggest that two significant concerns that arise through asking these questions are trust and truth. Drawing on over 50 years of social science research on trust[1], I suggest we can engage with AI systems to explore to what extent these systems provide the basic conditions for trust: does the system operate in line with our expectations of it (linking back to the ethical principle of confidence)? But we can go further and ask do we trust that the system will place our interests ahead of those who own or operate the system? We can also look at how trust is managed in demonstrations of AI and how AI disrupts the routine grounds of social order through which trust would normally persist.

With regards to truth, in the lecture I pose questions of the nature, source and reliance upon somewhat simplistic notions of truth that seem to pervade AI system development. I suggest this becomes problematic when assumptions are made that AI systems do no more than reflect truth that is already out there in the world independent of the technology. Without straying into debates about post-truth and its associated politics, it nonetheless seems problematic that systems with a generative capacity to create their own truth (at least to an extent) are then presented to the world as doing no more than re-presenting a truth that already exists independent of the system. In the lecture I also suggest that truth can be considered as an input (through the notion of ground truths that the system itself partially creates) and output (through the system’s results).

[1] For example, Barber’s (1983) work on trust, Shapin (1994), Garfinkel (1963)

Barber, B. (1983) The Logics and Limits of Trust (Rutgers University Press, NJ, USA)

Shapin, S. (1994) A Social History of Truth (University of Chicago Press, London)

Garfinkel, H. (1963) A conception of and experiments with ‘trust’ as a condition of stable concerted actions, in Harvey, O. (ed) Motivation and Social Interaction (Ronald Press, NY, USA) pp. 197-238

 

“I just remember the gasworks as big grey stone buildings that almost frightened you.” Connections with the past informs and inspires BDFI’s new research hub

University of Bristol historian Lena Ferriday concludes BDFI’s ‘History of the Sheds’ project with a summary of the living histories collected to inform its place the former industrial community of The Dings and St Philips. Alongside artist Ellie Shipman, Lena interviewed former local residents and employees of the Bristol Gas Company to uncover how it felt to live and work among a transformational industry for Bristol.

In June 2022, the BDFI became the first inhabitants of the new Temple Quarter Campus when they moved into their new building at 65 Avon Street. As part of this relocation, they were keen to investigate the histories of this site which once housed Bristol’s gasworks, in order to draw connections between the socio-technical pasts, presents and futures of the space. Following extensive archival research conducted by Dr James Watts, the two of us wrote a report which revealed the gasworks and gas industry’s important influences on Bristol in social, economic, environmental and technological terms. But we also identified that what was missing from the written record were the voices of those who lived and worked in the vicinity of this site, whose memories are obscured in the archives. As such, the project became a more participatory one.

Plan of Avon Street Gas Station, 1857. Bristol Archives, 28777/U/E/5/1.

After publicising a call for contributions, I conducted a set of oral history interviews with local residents who were all differently connected to the site of the gasworks. Concurrently, we commissioned illustrator and artist Ellie Shipman to work alongside us to produce an artistic response to the histories we were continuing to uncover and write. Ellie also engaged with members of the public through interviews and memory café chats – with the help of local historian and co-founder of the Barton Hill History Group Garry Atterton.[1] Across these conversations – both structured and less formal – themes highlighted in the previous report were brought out in animated, lively ways.[2] But people’s tangible memories of the site also took the history in new ways, sometimes with narratives that competed with the stories told by archival material.

Environment

Our original report had identified that the gasworks opened in 1821 and was frequently renovated until its decommissioning in 1970. In an interview with Geraldine Stone, who lived in East Bristol in the 1960s, she relayed that ‘I just remember the gasworks as big grey stone buildings that almost frightened you … cause they were again grey and dark’. This perspective of the building from the outside – as perhaps relational to the way in which the majority of Bristol’s inhabitants will experience the refurbished BDFI building – demonstrated the emotional power of the site on the memories of those who lived nearby. But they also showed the gasworks’ relationship to the other buildings in this industrial area. From the nineteenth to mid-twentieth centuries, this area of Bristol was fiercely industrial.

Neighbouring the gasworks were an ironworks, a vitriol works, a lead works, a paint works, a marble factory, a railway engine works, the railway and Temple Meads station, a timber yard and a dye factory. As Garry explained, ‘The Feeder Canal was sort of a good thing and a bad thing, in some respects. It was a good thing because it was an artery that spread the heart of Bristol out to other parts of the city … But because of the flat land that was available, it was brilliant for industry… What you saw in that time was a massive concentration of heavy industry that had not been seen in the city.’ Geraldine recalled that the area was ‘always dark, you know. But it was mostly the smells in everything that was going on, and because it was the centre of St Philips and The Dings, it was something that you lived with every day, because you didn’t have cars, you would walk everywhere.’ For Geraldine, living as a pedestrian in this area led to a particular form of sensory engagement with the industrial space, and also led to an understanding of Avon Street a transitional place: ‘To me Avon Street was a way of getting through to anywhere. It was a throughway.’

For Geraldine this dark, smelly area also produced a particular atmosphere in the area: ‘in those days when you had the trains and smoke, well in our days as a small child everything was smog. It was just smog. It used to come down really low and you couldn’t see…’ In the report we attested to the concern for the environmental impact of gas that was demonstrated nationally in the nineteenth century, and the moves within the Bristol Gas Company to reduce pollution.[3]  In his interview with Ellie, Garry corroborated this, writing that ‘You had this massive concentration there, which potentially was all going in the Feeder. All the pollutants.’ The report also attested to the company’s attempts to ensure this pollution did not cause medical concerns. Members of the Barton Hill Group recalled that those who worked in the area struggled with chest and lung issues – ‘coming home with really heavy coughs’ – although here Lysaghts steel works and the cotton factory were more frequently listed as embodied polluters than the gasworks.

Society

In the report we noted that this pollution made the gasworks an uncomfortable and challenging workplace. Richard Nicholls, whose father worked as a foreman at the site from 1944, noted that this physical atmosphere also had a social impact on the employees:

“It was unusual in those days for engineers to be able to talk about their marriage, their sex life or whatever. Because they were in what were dangerous conditions, you know, really hot there, the hot coals there and so on. So when they were in that condition they were very much a family of their own and looked after each other…”

Here, Richard attested to a community within the group of men working at the site that the archival records had shown to an extent, through evidence of an employee brass band and football team. Richard’s intergenerational memories bring this collegiality to life with stories. He noted that ‘it was very much a mick-taking humour’ between colleagues, recalling the instance in which ‘One of the guys had his shoes nailed down to the floor cause when they’d go to the building sites they’d change shoes, … and he had to go back in his muddy boots.’

It was the dangerous conditions that Richard also linked to the success of trade unionism within the industry: ‘that’s why the strikes were so solid, because they were family, they relied on each other, they could talk to each other about anything. They could speak their mind as it was. Very hard men, very stuck together.’ In our initial report we identified Bristol gas workers’ strong involvement in the strikes of 1866, 1889 and 1920, the archives attesting to the outcomes of this action. But the internal mentalities of those striking were not documented, and Richard’s comments are enlightening here.

Banners of the National Union of the Gas Workers and General Labourers Bristol District No 1 Branch. Credit to Bristol Museums, Galleries and Archives, T8389.

The successful 1889 strike saw a dispute over shift length, with workers petitioning for a move from 12 to 8-hour shifts. By the time Richard’s father was employed at Avon Street, shift patterns were fairly regular: ‘The normal one would be 8 till 5, 8 till 6, that sort of thing’. When engineers worked on call, however, they could be called out to emergency leaks across the city at any time. Richard noted that ‘it wasn’t a case of you said no. You went.’ The gas works were therefore closely connected to the insides of Bristolians’ homes in a way the archives had not accounted for, not only via the material substance of the gas but also those who monitored it. But it was not only engineers who brought the gasworks into the home. Members of the Barton Hill group recounted memories of company employees visiting to collect money from the gas meters, an exciting day as they would often get money back. Richard remembered ways in which people would also get around this system in certain ways: ‘there was a mechanism inside the meter where you could change the rate that you’d pay for your gas, and they used to get them to change it so that the money would go in … then when the meter reader came they’d make them a cup of tea, oh you’ve overpaid us this much, so they’d give them a refund out of the money that was in the thing’. Others would put coin-shaped objects into the meter to get themselves through the week until they confessed on collection day.

Technology

This connection into urban homes came with its difficulties. Archival records attested to concerns about the dangers of gas circulating from its introduction in the early nineteenth century, and numerous gas explosions were reported in the late nineteenth and early twentieth centuries. Members of the Barton Hill History Group similarly remembered a gas explosion that flattened two houses on Lincoln Street in the early 1950s.

Difficulties with gas not only arose from fears regarding its danger – from the 1890s acknowledgment of the efficiency and stability of electric light in industrial settings (beginning with the Wills Tobacco factory) posed a challenge to the Bristol gasworks. Yet gaslighting remained popular for longer than expected, and many of the participants in our project had strong recollections of it. When working as a gas engineer in Bristol, Richard recounted being called out to a house that still had no electricity in 1972: ‘She had gas lighting, gas cooking … So it’s amazing how long it was before some people had electrics.’ Richard noted being concerned about the hazards of the lighting here however: ‘the gas lights had little chains on that you could pull down to turn on and … She wanted me to lower it. She had big frizzy hair. And I thought, well if I lower that down, it’s going to be so near her hair – I could just see it going whoosh!’, he laughed.

Lamplighter, Bristol 1946, Bristol Archives 2877

The gasworks also contributed to energy in the home via other materials than gas. Members of the Barton Hill group recalled taking metal prams down to the gasworks to collect bags of coke for the fire – coke being a purified substance created in the production of coal gas. The elongated transition to electricity also shaped memories of gas itself, particularly given that gas is still used in many homes but now for cooking and central heating, rather than lighting. One member of the Barton Hill Group referred to cooking gas as ‘normal gas’, in comparison to the historic gas substance used to fuel lights. Yet in her interview Geraldine spoke of ‘the gas lamp posts being lit by somebody, by a man coming with a stick.’ Here too, however, her memories become sticky: ‘I don’t know if I imagined it – I didn’t and I know I didn’t …. so I think is that my vision or did I see it? But I’m sure I did …. Cause it used to shine all the time into my bedroom.’ Although she described the memory in detail, the chasm between gaslight technology and contemporary lighting innovations has worked to obscure her own visual childhood memories.

Conclusion

In the first report published on our findings about the site of Bristol gasworks, we concluded that ‘The tension between the benefit and harm brought by the introduction of gas to the city … is what characterises the history of this site.’ Predominantly the living histories we have collected have also pointed towards contested and competing narratives operating within the history of the gasworks. Gas was hazardous to the local landscape and residents’ bodies in the short term, with explosions destroying buildings and injuring inhabitants, and on a longer time scale, polluting the waterways and atmosphere which in turn brought on lung and chest issues for those immersed in them. Yet it also played a formative role in people’s fond memories of the area.

Given the BDFI’s emphasis on ensuring that the production of digital technology is inclusive and sustainable for the societies it affects, that the findings of these oral histories attest to the environmental, social and technological dimensions of the site is of great importance. Acknowledging via archival material that the former proprietors of the site also worked to reduce the environmental impact of innovative gas production, and that their employees campaigned for workplace equality provides a source of inspiration for the ways in with the Institute now inhabit this site in the present and into the future. The findings of the living and community histories that this new report has attested to, however, stretch beyond the site itself, and reaffirm the wider range of forms engagement with the gasworks site took. The gasworks had strong connections both to the wider industrial area of St Philips and to homes across the city, and in a similar way through this project the BDFI has maintained this connection between the sites and local communities through the sharing of memories. In addition, this project has amplified lived experience and as such demonstrated the importance of considering the individuals who are impacted by innovation in diverse ways.


Acknowledgements

The quotes in this piece are excerpted from recordings by Lena Ferriday with participants Geraldine Stone and Richard Nicholls, and Ellie Shipman with Garry Atterton, Pete Insole, and members of the Barton Hill History Group, Bern and Gill. We are very grateful to all those involved for sharing their time and memories with us, which have brought to life this historic site.

About the author

Lena FerridayLena is a PhD researcher in the Department of History here at Bristol, with an interest in modern histories of bodies, the environment and everyday experience. Her research has previously explored other areas of Bristol – including the city’s transport networks, tourism, green spaces and the history of the University – and she is currently writing a history of embodiment in nineteenth century Cornwall.

 

Notes

[1] A memory café is a community group that gathers people with shared memories (often of a place or event) to meet and discuss. The Barton Hill History Group run a monthly memory café, the August instalment of which Ellie joined.

[2] An edited audio piece including excerpts from these recordings was created by Ellie Shipman, and can be listened to here.

[3]This is also expanded in another blog post I have written to complement the project, considering the ways in which the senses profoundly shaped the urban production of gas.

Hopeful illustrations daring us to imagine sustainable energy innovations

In the midst of COP27 and as Europe-wide energy crisis BDFI seed-corn funding recipient Dr Ola Michalec describes how, as a social scientist, she has helped energy policy makers open up dialogue around of smarter energy systems through illustrations. In partnership with Bristol City Council, Ofgem and Energy Systems Catapult and communication and illustration experts she is helping to move the debate from the purely technical to social imaginings.

 

Winter is coming

We are at the cusp of winter, gearing up to spend more time indoors. As soon as the clocks returned to Greenwich Mean Time, basking in late-October sunlight seemed like a distant memory. To create a sense of homely “hygge”, I’ve recently completed a bi-annual reshuffle of the storage boxes. Christmas decorations, candles, and blankets have swapped places with beach hats, sandals, and camping gear. I’ve yet to put heating on, however. The biggest contributor to wintry cosiness comes at the highest cost, especially this year.

This sentiment resonates across the country, as individuals and businesses alike are nervously budgeting for the upcoming months. With so many complex questions arising, I’m grateful to the scientists and journalists for their excellent outreach and science communication. “Why has the price of gas increased?”, Rebecca Leber asks? “Who is profiting from my expensive bills?”, Graeme Demianyk explains.  “How to effectively target those at the highest risk of fuel poverty?”, Prof Aimee Ambrose discusses.

 The value of futures-thinking

With most of the public attention is directed to examining today’s crises, sometimes it might be challenging to imagine that many of us in the wider energy sector are working towards a greener, happier, and fairer future. Indeed, as a social scientist interested in emerging technologies, I occupy a peculiar space where the present and the future(s) meet.

At first, buzzwords like “open energy data”, “energy digitalisation” or “smart homes” might seem irrelevant to the current issues concerning energy affordability, sustainability, and security. Although emerging technologies will not help with our heating bills this winter, these seemingly futuristic visions of the “new power grid” are closer than we think. Various best practice guidance documents, standardisation proposals or regulatory consultations have sprouted over the UK energy policy landscape over the past several months. This is precisely why now is the best time to broaden the community of stakeholders and raise the important questions about the social implications of introducing novel energy technologies into our homes and infrastructure sites.

Meaningful participation of community organisations and citizens is critical for timely advancement of climate action (Stirling, 2008; Rommetveit et al, 2021). Infrastructures and policies, if introduced without the public approval, risk becoming delayed or rejected (see, for example, the troubling case of smart meters implementation programme – Michalec et al, 2019; Sovacool et al, 2017). However, engaging the lay public with complexity and autonomy of modern digital (or ‘smart’) systems has been proven challenging due to pre-requisite knowledge expected from the citizens (Pfotenhauer et al., 2019). Recently, scholars of energy systems and society argued for a participatory research agenda on energy systems digitalisation (Sareen, 2021 ). A paper co-authored by a fellow Bristol University researcher, Dr Caitlin Robinson, suggested five areas of further analysis and engagement: 1) the intersection of digital and financial inclusion; 2) social implications of flexibility 3) the role of trust in shaping engagement with innovations; 4) digital literacy and communications; 5) the uneven impacts of innovation on different social groups (Chambers et al., 2022).

Our research: on regulating smart energy appliances

In parallel to that, my research at the University of Bristol, explored the role of expertise in standardisation and policymaking initiatives in the context of smart energy (Michalec et al., in review) systems. We found that while there are numerous initiatives to facilitate the introduction of smart energy systems, they are usually framed as solely ‘technical’ projects which provide limited opportunities for engagement with citizens and community advocacy groups. However, security, privacy, and interoperability of energy data are inherently socio-technical considerations that necessitate opening up of the public debate.

Smart lens: Sketching new perspectives on energy systems

Thanks to seed-corn funding from Bristol Digital Futures Institute, we were able to fund a project exploring illustration as a medium of public engagement in energy futures. First, we assembled a team of researchers (Dr Ruzanna Chitchyan and I), an illustrator (Oliver Dean), a science communication expert (Dr Emma Osborne) and industry partners (Bristol City Council, Ofgem and Energy Systems Catapult). Second, we worked collaboratively to produce briefs for the artist. Third, we engaged in several rounds of sketching and feedback until we reached a version we were all happy with. Et voilá! Let me introduce you to a few of our illustrations (you can find a full set here).

Bristol 2030

This is a series of eight images (available as postcards or a large poster) depicting digital energy innovations in several iconic Bristol locations: from Ashton Gate Stadium, Millennium Square to Easton Community Centre, among the others. We wanted to show a variety of communities celebrating sustainable, inclusive, and optimistic futures. While most of media reporting focuses on scaremongering and fatalistic accounts, we created images that could be used as conversation-starters for more hopeful discussions. Our postcards were displayed during an exhibition in We The Curious earlier this year.

Old Grid/New Grid

Old grid/ New Grid is a prototype card game. With 28 images representing technological, social, and regulatory aspects of the energy systems, we designed a structured activity for schools and community energy organisations. So far, we have received interest from organisations such as Bath and West Community Energy Cooperative and are always keen to hear from other potential collaborators!

Security and data sharing platforms: get on the right track!

One of our project partners, Energy Systems Catapult, requested an attractive infographic aimed at people working in the energy sector without background in cyber security. Energy Systems Catapult is a supporter of the Open Data paradigm in the industry but faced difficulties with communicating it in an accessible way, while resolving misunderstandings around security and privacy. We came up with a metaphor of a tube station to show that open data is about re-considering who should access various information, rather than publishing all datasets freely accessible websites. The infographic is now used for Catapult’s onboarding workshops and other events.

What’s next?

 Public engagement is never a ‘finished job’ – there are always new stakeholders to meet and new issues to discuss. That said, as researchers, we often tend to side-line these activities; it is difficult (if not impossible!) to directly demonstrate impact (echoing my reflections on policy engagement for the Cabot Institute blog). What would a measure of impact look like in this case anyway? A number of people who wrote to their MPs about sustainable energy, who threw soup at a painting in Tate, who attached solar panels to their cats? While I am not planning on tracking any of the above, let me tell you about our future plans…

  • I’ll be presenting a lunchtime talk for the REPHRAIN National Research Centre on Privacy, Harm Reduction and Adversarial Influence Online on the 17th Nov 1-2pm (see image for joining instructions). I will be discussing “Using creative methods to engage people in cyber security conversations”

 

 

 

 

  • I am preparing a public engagement workshop using our wonderful “postcards from Bristol 2030”. This will be an interactive, local, and open event aimed at all Bristolians (whether native or adopted) interested in sustainable futures of the city. Time and place TBC
  • I would love an opportunity to exhibit the images – perhaps in your city or your community? Please let me know if you could introduce me to relevant people! Contact me.

Convoluting poetry + maths – Poetrishy explores the possibilities

BDFI seedcorn-funded researcher Dr Rebecca Kosick explains how her project, Poetrishy has taken off and where the collaborations between the worlds of poetry and maths might lead.  Check out the stylish editions that can also be purchased in print from Tangent Books.

  • How ambitious is Poetrishy?

Poetrishy has big ambitions in that it is trying to bring together two fields of practice—poetry and maths—that are not obvious allies. We have found ways they can be, but our most ambitious goal, which we aren’t sure if we have yet realized, has to do with the convolution of these two fields.

Convolution increases the challenge, in that rather than just igniting an encounter between maths and poetry, we are trying generate opportunities for the two fields to influence and mutually modify each other, creating something new in the process. For our second edition, we spent a lot of time reflecting on the ways we have seen mathematics influencing poetry, particularly by creating new forms and possibilities for poetic production. This direction of influence is pretty well established in our experiments, and builds on earlier work we did in collaboration with the Brigstow Institute. The opposite direction, where poetry can influence and modify maths, seems to be still a nascent and more speculative possibility, though we have some ideas. For instance, my collaborator Mauro Fazion is working with other researchers to look into how metaphor and meaning-making in poetry can inform mathematical modelling of lexical and semantic evolution. Here we think poetry may have something to contribute to mathematics and its applications. And we are eager to see what other possibilities are out there.

  • What do the works submitted so far, tell us about our digital futures?

It’s been really interesting to see the range of submissions we have gotten, and to discover that the community of people interested in poetry and maths is bigger than you might expect. For me, this speaks to the continued vitality of the arts during the so-called digital age.

Plosive Consonants by Bruno Ministro for Poetrishy #1

I don’t think this was ever really in doubt for artists, or for those of us who study contemporary arts and humanities, but we still see ways in which the STEM disciplines are understood as, on the one hand, distinct from the arts, and on the other, as having a special claim on technology that the arts somehow don’t have. I think we can contest this claim historically, and with an eye toward the future too. Poets, in particular, have been keen to explore the possibilities that new technologies enable for the creation and dissemination of poetry, from the typewriter to the mimeograph and the algorithmic computing. I expect this will continue and that there are surprises yet for us to discover.

  • As this is an evolving project, what adjustments have you made along the way? What have been the most challenging aspects, and the most surprising?

One of the more technical challenges had was to do with how to display the range of poetic materials we were receiving (and publishing) in Poetrishy. We honestly didn’t know what to expect when we put out our first call, in that we were open to all kinds of formal possibilities, from text-based lyrical poems to apps, interactive web-based tools, videos, and more. We ended up receiving a range of submissions that exceeded, even, our own open imagination of the parameters of what we might expect. And then we needed to figure out how to first, share these works via some kind of unified digital platform, and second, share them in print form.

Our designers, Russell Britton (web designer) and Johanna Darque (print designer and co-editor), did such a fantastic job of bringing together a huge diversity of contributions, and in navigating the affordances and needs of digital versus print publishing. You should definitely check out both versions, digital and print. On top of making gorgeous and innovative homes for Poetrishy in each of these platforms, the team also worked hard to build a kind of flexible reciprocity between the web and paper versions, producing what Jo Darque called “non-identical twins.” The web version and the print journal are each their own distinct but linked elaborations of what Poetrishy​​​​ is.

  • When will we see the next edition?

We are working on the print version of Poetrishy #2 now (Autumn 2022), and it should be available for sale in the coming months. We are hoping to continue publishing Poetrishy in the coming years as well and will be looking for funding to make this happen. We are grateful to the BDFI for believing in this project and helping us get it off the ground.

 

Poetrishy is published by a team of poets, mathematicians, editors, and designers: Mauro Fazion, Rebecca Kosick, Rowan Evans, Ademir Demarchi, Miranda Lynn Barnes, Johanna Darque, and Russell Britton.

Professor Susan Halford reflects on leading BDFI as she steps down as co-director

Professor Susan Halford and Professor Dimitra Simeondiou have led the development of BDFI since 2019. Here Susan reflects on what has made the endeavour special to her as she announces that she is standing down as academic co-Director.  Susan will continue to work with BDFI supporting its activities and mission and as the key link to the new ESRC Centre for Sociodigital Futures.

It has been a remarkable privilege to co-lead BDFI in these formative years and I am very proud indeed of what Dimitra, myself and the team have achieved during this time.

What started as a hugely ambitious aim in 2018 – to do digital futures differently – is now a unique and lively Institute, located in beautifully refurbished Victorian ‘sheds’ which will house state-of-the-art innovative research facilities for academics from across the university and our partners in industry, government and civil society.

Breaking the linear model of innovation

During the last four years I’ve been continually impressed by the commitment of our partners and academics to transform the way we do digital innovation. This calls on us all to break the linear model of innovation – where technology comes first, and impact is examined later – and to develop sociotechnical thinking, methodologies and innovation practices. This is far from trivial, and we have learned a lot along the way, but BDFI is uniquely placed to succeed because we are truly interdisciplinary, starting with the leadership and reaching right across the Institute.

Strong foundations

BDFI now sits at the heart of a truly impressive stream of sociotechnical research and innovation at the University of Bristol.

Susan Halford
Susan Halford at the new research hub in Temple Quarter Enterprise Campus

We have laid the foundations for a step-change in how we think about and do digital innovation. With ten outstanding new academic appointments already in place, and ten more to come, there is no doubt that BDFI will go from strength to strength. I look forward to remaining actively involved as the institute grows and drives digital innovation for fair, sustainable and prosperous futures.

Explaining AI decision making: A sociotechnical approach

Dr Marisela Gutierrez Lopez has been collaborating with BDFI partner LV=General Insurance to explore opening up processes behind AI decision making. How will this benefit organisations and people who are affected by automated decisions?

Sociotechnical methods are helping us to create more inclusive ways of AI discussion across public, private and academic sectors. They are therefore crucial to investigate how people shape and are shaped by AI systems, and explore the interrelations between people, algorithms, data, organisational procedures and other factors that constitute these systems. For this purpose, we integrate social and technological expertise from across the University of Bristol, and our partners in industry and communities to empirically examine what makes AI explainable from a sociotechnical perspective.

In July 2020, the Explaining AI project was started with the vision to examine the concept of “Explainable AI” (or XAI) in machine learning approaches used in decision making. Our aim was to move beyond technocratic perspectives where explanations are framed as technical challenges towards more inclusive approaches that consider what AI might mean for diverse data publics – particularly those not usually included in discussions about AI or explainability.

Working collaboratively with LV= General Insurance (LV= GI), a leading insurance provider in the UK, we are investigating the different levels of explanation of the decision-making processes informed by machine learning models and their outcomes. In addition to our investigations at a commercial setting, we have also teamed up with two local partners – Black South West Network and Knowle West Media Centre – to explore the types of explanations that would make machine learning intelligible and actionable to these communities.

Reaching out to local communities

The community strand of our project is underpinned by design justice as a framework for reconstructing Explainable AI in collaboration with those at the margins of innovation. We avoid positioning ourselves as outsiders that tell communities what AI is or why it matters. We are not aiming to solve to the black-box problem. Instead, we start from the “bottom-up”, exploring community interests and concerns as a first step.

We are co-producing community-led XAI initiatives with our community partners to ensure machine learning decisions are communicated in relatable and actionable ways. This has given our partners ownership over the project and its outcomes. For example, each community partner is shaping up their initiative by defining their research questions and the focus of their community engagements.

woman speaking in a groupThese community-led initiatives allow for open and speculative conversations that generate knowledge (in opposition to traditional forms of XAI), moving from individual to community understandings of what constitutes AI, and shifting the focus of attention from the past and present to possible futures. The next steps of our project involve supporting community engagements by the community groups to reach into their local areas and produce new XAI approaches that empower and give agency to different data publics.

Embedding our research at LV= GI

For the organisational strand, we set up a participatory ethnography where BDFI researchers are embedded in the LV= General Insurance data science team. As a result, the project offers a unique opportunity to closely analyse organisational practices and ways of working between data science and other business functions.

This project allows us to collectively explore ways to explain machine learning models beyond providing technical accounts of data and complying with legal requirements. It shifts the perception on what makes AI explainable with an enhanced understanding of how machine learning is shaping the organisation. Moreover, it has given us, both the research team and LV= GI practitioners, space to form deep connections, share co-working spaces, and expand our partnership even further.

Putting together industry and community – XAI perspectives 

Our project responds to the current ethical turn in AI by disrupting the concept of explainability, moving away from a purely technical solution to explaining the practice of AI rather than the principle itself. Sociotechnical methods are helping us to make research results actionable, where outputs are not abstract or distant but directly applicable in the context of each project partner.

The knowledge and dynamics generated using these methods are also helping us to connect the outcomes of the organisational and community strands of the project. Putting together cross-sector collaborations in XAI involves mutual learning, where the perspectives of all partners are equally important, and we learn from each other strengths. Additionally, it requires flexibility to adjust our priorities and facilitate two-way conversations. These conversations will become crucial in the last year of the project as we reconstruct Explainable AI together, in consideration of the findings from each place of inquiry. This will allow us to create more inclusive processes for the development of machine learning in the future.