Hopeful illustrations daring us to imagine sustainable energy innovations

In the midst of COP27 and as Europe-wide energy crisis BDFI seed-corn funding recipient Dr Ola Michalec describes how, as a social scientist, she has helped energy policy makers open up dialogue around of smarter energy systems through illustrations. In partnership with Bristol City Council, Ofgem and Energy Systems Catapult and communication and illustration experts she is helping to move the debate from the purely technical to social imaginings.


Winter is coming

We are at the cusp of winter, gearing up to spend more time indoors. As soon as the clocks returned to Greenwich Mean Time, basking in late-October sunlight seemed like a distant memory. To create a sense of homely “hygge”, I’ve recently completed a bi-annual reshuffle of the storage boxes. Christmas decorations, candles, and blankets have swapped places with beach hats, sandals, and camping gear. I’ve yet to put heating on, however. The biggest contributor to wintry cosiness comes at the highest cost, especially this year.

This sentiment resonates across the country, as individuals and businesses alike are nervously budgeting for the upcoming months. With so many complex questions arising, I’m grateful to the scientists and journalists for their excellent outreach and science communication. “Why has the price of gas increased?”, Rebecca Leber asks? “Who is profiting from my expensive bills?”, Graeme Demianyk explains.  “How to effectively target those at the highest risk of fuel poverty?”, Prof Aimee Ambrose discusses.

 The value of futures-thinking

With most of the public attention is directed to examining today’s crises, sometimes it might be challenging to imagine that many of us in the wider energy sector are working towards a greener, happier, and fairer future. Indeed, as a social scientist interested in emerging technologies, I occupy a peculiar space where the present and the future(s) meet.

At first, buzzwords like “open energy data”, “energy digitalisation” or “smart homes” might seem irrelevant to the current issues concerning energy affordability, sustainability, and security. Although emerging technologies will not help with our heating bills this winter, these seemingly futuristic visions of the “new power grid” are closer than we think. Various best practice guidance documents, standardisation proposals or regulatory consultations have sprouted over the UK energy policy landscape over the past several months. This is precisely why now is the best time to broaden the community of stakeholders and raise the important questions about the social implications of introducing novel energy technologies into our homes and infrastructure sites.

Meaningful participation of community organisations and citizens is critical for timely advancement of climate action (Stirling, 2008; Rommetveit et al, 2021). Infrastructures and policies, if introduced without the public approval, risk becoming delayed or rejected (see, for example, the troubling case of smart meters implementation programme – Michalec et al, 2019; Sovacool et al, 2017). However, engaging the lay public with complexity and autonomy of modern digital (or ‘smart’) systems has been proven challenging due to pre-requisite knowledge expected from the citizens (Pfotenhauer et al., 2019). Recently, scholars of energy systems and society argued for a participatory research agenda on energy systems digitalisation (Sareen, 2021 ). A paper co-authored by a fellow Bristol University researcher, Dr Caitlin Robinson, suggested five areas of further analysis and engagement: 1) the intersection of digital and financial inclusion; 2) social implications of flexibility 3) the role of trust in shaping engagement with innovations; 4) digital literacy and communications; 5) the uneven impacts of innovation on different social groups (Chambers et al., 2022).

Our research: on regulating smart energy appliances

In parallel to that, my research at the University of Bristol, explored the role of expertise in standardisation and policymaking initiatives in the context of smart energy (Michalec et al., in review) systems. We found that while there are numerous initiatives to facilitate the introduction of smart energy systems, they are usually framed as solely ‘technical’ projects which provide limited opportunities for engagement with citizens and community advocacy groups. However, security, privacy, and interoperability of energy data are inherently socio-technical considerations that necessitate opening up of the public debate.

Smart lens: Sketching new perspectives on energy systems

Thanks to seed-corn funding from Bristol Digital Futures Institute, we were able to fund a project exploring illustration as a medium of public engagement in energy futures. First, we assembled a team of researchers (Dr Ruzanna Chitchyan and I), an illustrator (Oliver Dean), a science communication expert (Dr Emma Osborne) and industry partners (Bristol City Council, Ofgem and Energy Systems Catapult). Second, we worked collaboratively to produce briefs for the artist. Third, we engaged in several rounds of sketching and feedback until we reached a version we were all happy with. Et voilá! Let me introduce you to a few of our illustrations (you can find a full set here).

Bristol 2030

This is a series of eight images (available as postcards or a large poster) depicting digital energy innovations in several iconic Bristol locations: from Ashton Gate Stadium, Millennium Square to Easton Community Centre, among the others. We wanted to show a variety of communities celebrating sustainable, inclusive, and optimistic futures. While most of media reporting focuses on scaremongering and fatalistic accounts, we created images that could be used as conversation-starters for more hopeful discussions. Our postcards were displayed during an exhibition in We The Curious earlier this year.

Old Grid/New Grid

Old grid/ New Grid is a prototype card game. With 28 images representing technological, social, and regulatory aspects of the energy systems, we designed a structured activity for schools and community energy organisations. So far, we have received interest from organisations such as Bath and West Community Energy Cooperative and are always keen to hear from other potential collaborators!

Security and data sharing platforms: get on the right track!

One of our project partners, Energy Systems Catapult, requested an attractive infographic aimed at people working in the energy sector without background in cyber security. Energy Systems Catapult is a supporter of the Open Data paradigm in the industry but faced difficulties with communicating it in an accessible way, while resolving misunderstandings around security and privacy. We came up with a metaphor of a tube station to show that open data is about re-considering who should access various information, rather than publishing all datasets freely accessible websites. The infographic is now used for Catapult’s onboarding workshops and other events.

What’s next?

 Public engagement is never a ‘finished job’ – there are always new stakeholders to meet and new issues to discuss. That said, as researchers, we often tend to side-line these activities; it is difficult (if not impossible!) to directly demonstrate impact (echoing my reflections on policy engagement for the Cabot Institute blog). What would a measure of impact look like in this case anyway? A number of people who wrote to their MPs about sustainable energy, who threw soup at a painting in Tate, who attached solar panels to their cats? While I am not planning on tracking any of the above, let me tell you about our future plans…

  • I’ll be presenting a lunchtime talk for the REPHRAIN National Research Centre on Privacy, Harm Reduction and Adversarial Influence Online on the 17th Nov 1-2pm (see image for joining instructions). I will be discussing “Using creative methods to engage people in cyber security conversations”





  • I am preparing a public engagement workshop using our wonderful “postcards from Bristol 2030”. This will be an interactive, local, and open event aimed at all Bristolians (whether native or adopted) interested in sustainable futures of the city. Time and place TBC
  • I would love an opportunity to exhibit the images – perhaps in your city or your community? Please let me know if you could introduce me to relevant people! Contact me.

Convoluting poetry + maths – Poetrishy explores the possibilities

BDFI seedcorn-funded researcher Dr Rebecca Kosick explains how her project, Poetrishy has taken off and where the collaborations between the worlds of poetry and maths might lead.  Check out the stylish editions that can also be purchased in print from Tangent Books.

  • How ambitious is Poetrishy?

Poetrishy has big ambitions in that it is trying to bring together two fields of practice—poetry and maths—that are not obvious allies. We have found ways they can be, but our most ambitious goal, which we aren’t sure if we have yet realized, has to do with the convolution of these two fields.

Convolution increases the challenge, in that rather than just igniting an encounter between maths and poetry, we are trying generate opportunities for the two fields to influence and mutually modify each other, creating something new in the process. For our second edition, we spent a lot of time reflecting on the ways we have seen mathematics influencing poetry, particularly by creating new forms and possibilities for poetic production. This direction of influence is pretty well established in our experiments, and builds on earlier work we did in collaboration with the Brigstow Institute. The opposite direction, where poetry can influence and modify maths, seems to be still a nascent and more speculative possibility, though we have some ideas. For instance, my collaborator Mauro Fazion is working with other researchers to look into how metaphor and meaning-making in poetry can inform mathematical modelling of lexical and semantic evolution. Here we think poetry may have something to contribute to mathematics and its applications. And we are eager to see what other possibilities are out there.

  • What do the works submitted so far, tell us about our digital futures?

It’s been really interesting to see the range of submissions we have gotten, and to discover that the community of people interested in poetry and maths is bigger than you might expect. For me, this speaks to the continued vitality of the arts during the so-called digital age.

Plosive Consonants by Bruno Ministro for Poetrishy #1

I don’t think this was ever really in doubt for artists, or for those of us who study contemporary arts and humanities, but we still see ways in which the STEM disciplines are understood as, on the one hand, distinct from the arts, and on the other, as having a special claim on technology that the arts somehow don’t have. I think we can contest this claim historically, and with an eye toward the future too. Poets, in particular, have been keen to explore the possibilities that new technologies enable for the creation and dissemination of poetry, from the typewriter to the mimeograph and the algorithmic computing. I expect this will continue and that there are surprises yet for us to discover.

  • As this is an evolving project, what adjustments have you made along the way? What have been the most challenging aspects, and the most surprising?

One of the more technical challenges had was to do with how to display the range of poetic materials we were receiving (and publishing) in Poetrishy. We honestly didn’t know what to expect when we put out our first call, in that we were open to all kinds of formal possibilities, from text-based lyrical poems to apps, interactive web-based tools, videos, and more. We ended up receiving a range of submissions that exceeded, even, our own open imagination of the parameters of what we might expect. And then we needed to figure out how to first, share these works via some kind of unified digital platform, and second, share them in print form.

Our designers, Russell Britton (web designer) and Johanna Darque (print designer and co-editor), did such a fantastic job of bringing together a huge diversity of contributions, and in navigating the affordances and needs of digital versus print publishing. You should definitely check out both versions, digital and print. On top of making gorgeous and innovative homes for Poetrishy in each of these platforms, the team also worked hard to build a kind of flexible reciprocity between the web and paper versions, producing what Jo Darque called “non-identical twins.” The web version and the print journal are each their own distinct but linked elaborations of what Poetrishy​​​​ is.

  • When will we see the next edition?

We are working on the print version of Poetrishy #2 now (Autumn 2022), and it should be available for sale in the coming months. We are hoping to continue publishing Poetrishy in the coming years as well and will be looking for funding to make this happen. We are grateful to the BDFI for believing in this project and helping us get it off the ground.


Poetrishy is published by a team of poets, mathematicians, editors, and designers: Mauro Fazion, Rebecca Kosick, Rowan Evans, Ademir Demarchi, Miranda Lynn Barnes, Johanna Darque, and Russell Britton.

Professor Susan Halford reflects on leading BDFI as she steps down as co-director

Professor Susan Halford and Professor Dimitra Simeondiou have led the development of BDFI since 2019. Here Susan reflects on what has made the endeavour special to her as she announces that she is standing down as academic co-Director.  Susan will continue to work with BDFI supporting its activities and mission and as the key link to the new ESRC Centre for Sociodigital Futures.

It has been a remarkable privilege to co-lead BDFI in these formative years and I am very proud indeed of what Dimitra, myself and the team have achieved during this time.

What started as a hugely ambitious aim in 2018 – to do digital futures differently – is now a unique and lively Institute, located in beautifully refurbished Victorian ‘sheds’ which will house state-of-the-art innovative research facilities for academics from across the university and our partners in industry, government and civil society.

Breaking the linear model of innovation

During the last four years I’ve been continually impressed by the commitment of our partners and academics to transform the way we do digital innovation. This calls on us all to break the linear model of innovation – where technology comes first, and impact is examined later – and to develop sociotechnical thinking, methodologies and innovation practices. This is far from trivial, and we have learned a lot along the way, but BDFI is uniquely placed to succeed because we are truly interdisciplinary, starting with the leadership and reaching right across the Institute.

Strong foundations

BDFI now sits at the heart of a truly impressive stream of sociotechnical research and innovation at the University of Bristol.

Susan Halford
Susan Halford at the new research hub in Temple Quarter Enterprise Campus

We have laid the foundations for a step-change in how we think about and do digital innovation. With ten outstanding new academic appointments already in place, and ten more to come, there is no doubt that BDFI will go from strength to strength. I look forward to remaining actively involved as the institute grows and drives digital innovation for fair, sustainable and prosperous futures.

Explaining AI decision making: A sociotechnical approach

Dr Marisela Gutierrez Lopez has been collaborating with BDFI partner LV=General Insurance to explore opening up processes behind AI decision making. How will this benefit organisations and people who are affected by automated decisions?

Sociotechnical methods are helping us to create more inclusive ways of AI discussion across public, private and academic sectors. They are therefore crucial to investigate how people shape and are shaped by AI systems, and explore the interrelations between people, algorithms, data, organisational procedures and other factors that constitute these systems. For this purpose, we integrate social and technological expertise from across the University of Bristol, and our partners in industry and communities to empirically examine what makes AI explainable from a sociotechnical perspective.

In July 2020, the Explaining AI project was started with the vision to examine the concept of “Explainable AI” (or XAI) in machine learning approaches used in decision making. Our aim was to move beyond technocratic perspectives where explanations are framed as technical challenges towards more inclusive approaches that consider what AI might mean for diverse data publics – particularly those not usually included in discussions about AI or explainability.

Working collaboratively with LV= General Insurance (LV= GI), a leading insurance provider in the UK, we are investigating the different levels of explanation of the decision-making processes informed by machine learning models and their outcomes. In addition to our investigations at a commercial setting, we have also teamed up with two local partners – Black South West Network and Knowle West Media Centre – to explore the types of explanations that would make machine learning intelligible and actionable to these communities.

Reaching out to local communities

The community strand of our project is underpinned by design justice as a framework for reconstructing Explainable AI in collaboration with those at the margins of innovation. We avoid positioning ourselves as outsiders that tell communities what AI is or why it matters. We are not aiming to solve to the black-box problem. Instead, we start from the “bottom-up”, exploring community interests and concerns as a first step.

We are co-producing community-led XAI initiatives with our community partners to ensure machine learning decisions are communicated in relatable and actionable ways. This has given our partners ownership over the project and its outcomes. For example, each community partner is shaping up their initiative by defining their research questions and the focus of their community engagements.

woman speaking in a groupThese community-led initiatives allow for open and speculative conversations that generate knowledge (in opposition to traditional forms of XAI), moving from individual to community understandings of what constitutes AI, and shifting the focus of attention from the past and present to possible futures. The next steps of our project involve supporting community engagements by the community groups to reach into their local areas and produce new XAI approaches that empower and give agency to different data publics.

Embedding our research at LV= GI

For the organisational strand, we set up a participatory ethnography where BDFI researchers are embedded in the LV= General Insurance data science team. As a result, the project offers a unique opportunity to closely analyse organisational practices and ways of working between data science and other business functions.

This project allows us to collectively explore ways to explain machine learning models beyond providing technical accounts of data and complying with legal requirements. It shifts the perception on what makes AI explainable with an enhanced understanding of how machine learning is shaping the organisation. Moreover, it has given us, both the research team and LV= GI practitioners, space to form deep connections, share co-working spaces, and expand our partnership even further.

Putting together industry and community – XAI perspectives 

Our project responds to the current ethical turn in AI by disrupting the concept of explainability, moving away from a purely technical solution to explaining the practice of AI rather than the principle itself. Sociotechnical methods are helping us to make research results actionable, where outputs are not abstract or distant but directly applicable in the context of each project partner.

The knowledge and dynamics generated using these methods are also helping us to connect the outcomes of the organisational and community strands of the project. Putting together cross-sector collaborations in XAI involves mutual learning, where the perspectives of all partners are equally important, and we learn from each other strengths. Additionally, it requires flexibility to adjust our priorities and facilitate two-way conversations. These conversations will become crucial in the last year of the project as we reconstruct Explainable AI together, in consideration of the findings from each place of inquiry. This will allow us to create more inclusive processes for the development of machine learning in the future.

The Streets Seen and ‘The Sheds’ Smelled

Lena Ferriday is co-author of ‘Avon Street Gasworks and Bristol’s Gas Industry‘ with Dr James Watts, a report commissioned for BDFI to examine the histories of their renovated industrial building in St Philips, Bristol.  Here she looks at the senses most provoked by the production and distribution of gas – sight and smell.

In 1861, the Bristol Mirror proclaimed that,

Of all the social improvements that the last 50 years have seen brought about, none is more significant of progress than the lighting with gas all the thoroughfares of our towns. To look back to the year 1800 in this respect and conceive what the streets […] of our own city […] were after dark, without the aid of gaslight, is a task most of us would rather shirk than encounter. Yet there are those living and walking in our midst […] who can very easily go back in memory to the period when a light was shed upon the darkness that prevailed by Winsor illuminating the metropolis.[1]

 For historians of nineteenth century dark and light, the introduction of gaslighting was a revelation for urban life, stimulating a new economy where the hours of factory work and public leisure time were able to extend into the evening without the sun’s aid.[3] For Constance Classen, this innovation ‘blurred the age-old sensory divide between the visuality of daytime and the tactility of night-time’.[4] In the streets, this was indeed true. Yet as this short piece will show, in the industrial setting and on the level of company organisation, it was the interaction of sight with another sense, that of smell, that proved most important.

Lamplighter, Bristol 1946, Bristol Archives 2877

The first gas works was established in London in 1814. By 1819 gas works were in operation throughout the country, and in the mid-1820s most big cities were supplied with gas. The Bristol Gas Light Company first manifested in 1815 and was incorporated in 1818, working to produce coal-gas for the purposes of lighting. The first gas lamps were lit in the city, inside the Exchange, and on Wine and St. Nicholas Streets. Looking back, Bristol’s press proclaimed that from the Gas Light Company’s formation ‘we have had light shed upon our doings when the orb of day has sunk beneath the horizon, which, though it may not equal that furnished by the sun […] is yet the best substitute that has ever been discovered.’[2]

In 1821, the company headquarters expanded beyond its site at Temple Back and was rehoused to 65 Avon Street, in the building now known as ‘The Sheds’. Coal-gas was produced through a burning of coal to distil it into coke and capturing the gas that this produced, and the ‘Sheds’ was comprised of the Coal Shed, for storage, and Retort House, which the oven heating the coal to release gas, as seen in Fig. 2.

Plan of Avon Street Gas Station, 1857. Bristol Archives, 28777/U/E/5/1.

This production of gas did, however, have notoriously sensory consequences. This was not least in the noxious odours that were emitted from coal-gas as a result of its containing Hydrogen sulphide, characterised by a distinctively sulphuric scent. Despite commenting in 1861 that ‘These lights are clear, white, and beautiful – luminous, without any smoke or obnoxous effluvium, producing an effect equal to daylight, at about one-third the expense usually employed to obtain a miserable substitute’, John Breillat, one of the Gas Company’s original engineers, worked with his team across the 1820s in an attempt to reduce the smell of the gas production, and its pollution of the local area.[5] Local inhabitants also complained of the smell and in its early years calls were made for the Company to switch to the use of oil gas. extracted from whale or seal blubber.

These attempts failed, however, as the company’s management committee decided that to produce ‘the same quantity of light’ oil gas was significantly more expensive than coal, at a ratio of roughly 5:3.[6] To some extent then, sight was prioritised above smell: the importance of a gas emitting strong light to combat the darkness more important than the strength of its odour. Yet as part of their ‘exposition’ of the oil gas scheme, Bristol Mirror also concluded that ‘the ridiculous assertion of Oil-Gas being without smell, is also without foundation’, having found evidence that oil gas too ‘invaded’ homes with ‘a most distressing stench’.[7]

The failure to institute changes within the Gas Light Company led to a movement that formed a separate oil gas company, which by August 1823 had instated the Bristol and Clifton Oil Gas Company Act forbidding the company from using coal gas. Yet as the cost of whale oil rose in the 1830s, price differentiation once again took precedent over olfactory adversity, and the Oil Gas Company also began to use coal gas in 1836. For nearly two decades the companies operated alongside one another, each serving different Bristolian districts, until they were finally amalgamated in 1853 as the Bristol United Gaslight Company.

Whilst the changes to visual experience have commonly been seen as the key indicator of gaslight innovation’s sensory influence on urban space, in the case of Bristol’s Gasworks’ civic and industrial position the eye was not entirely dominant. For the city’s inhabitants, the gasworks were a strong-smelling presence and this sensory characteristic had great impact on the Gas Company’s bureaucratic and manufacturing development in its early years.


[1] ‘Jubilee of Gas Lighting in Bristol’, Bristol Mercury, 7 Sept 1861, p.4.

[2] ‘Jubilee of Gas Lighting in Bristol’, Bristol Mercury, 7 Sept 1861, p.4.

[3] Wolfgang Schivelbusch, Disenchanted Night: The Industrialization of Light in the Nineteenth Century, trans. Angela Davies (University of California Press, 1995) 16; Lynda Nead, Victorian Babylon: People, Streets and Images in Nineteenth-Century London (Yale University Press, 2000) p.98. Further, see Chris Otter, The Victorian Eye: A Political History of Light and Vision in Britain, 1800-1910 (University of Chicago Press, 2008).

[4] Constance Classen, ‘Introduction: The Transformation of Perception‘ in Constance Classen (ed.), A Cultural History of the Senses in the Age of Empire (London: Bloomsbury, 2014) 8-10.

[5] ‘Jubilee of Gas Lighting in Bristol’, Bristol Mercury, 7 Sept 1861, p.4.

[6] ‘Exposition of the Oil Gas Scheme’, Bristol Mirror, 3 March 1823, p.3.

[7] ‘Exposition of the Oil Gas Scheme’, Bristol Mirror, 3 March 1823, p.3.

Making play-based maths easier for teachers to assess – testing a blend of low and hi tech approaches

Michael Rumbelow and Professor Alf Coles lead one of our seedcorn-funded projects that aims to help boost children’s confidence in maths.

Using an AI driven app, the interchange between learning is explored through traditional use of blocks.  Here they discuss how digitising this learning aid could benefit teacher classroom assessment and the challenges of developing novel technologies as education specialists.

In 1854, the first English-speaking Kindergarten opened in London, based on the play-based pedagogy of Friedrich Froebel (1782-1852), who designed his Kindergarten curriculum around play activities with wooden blocks. Later plastic versions of Froebel’s blocks were developed, which evolved into Lego – now the world’s largest toymaker – as well as into interlocking plastic cubes for primary mathematics classrooms – which the characters in the popular CBeebies cartoon series Numberblocks are made of. And more recently, free play with digital cubes became the basis of Minecraft, the most popular video game of all time.

Figure 1. Sketches of using wooden cubes to model halving and quartering from an 1855 Kindergarten handbook.

Clearly, block play is a popular activity among children. And in schools there has also been a resurgence in the use of physical blocks in primary mathematics classrooms, following the government’s policy since 2016 of promoting so-called ‘Asian mastery’ approaches to teaching maths, as used in Singapore, China, South Korea and Japan, which make extensive use of physical blocks as concrete models of abstract mathematical concepts, such as counting, addition, multiplication etc. We were interested in researching children’s interactions with physical blocks from a mathematics education perspective, and one of the key challenges was how to capture data on children’s interactions with blocks for analysis.

Previous studies of block play have focused on gathering data variously through sketching or taking photos or videos of children’s block constructions, or embedding radio transmitters in blocks which could transmit their positions and orientations. Recently developments in computer vision technology offer novel ways of capturing data on block play. For example, photogrammetry apps such as 3D Scanner can now create 3D digital models from images or video of objects taken on mobile phones, and AI-based object recognition apps are increasingly able to detect objects they have been trained to ‘see’.

We felt there might be an opportunity to detect and digitise the positions of wooden or plastic cubes on a tabletop directly through a webcam, so that the coordinates of the corners could be used to create virtual animated models of stages of block constructions which could then be explored in various ways, such as in immersive virtual 3D environments, by both researchers and students. This abstracted coordinate data would also enable patterns of real-world block constructions to then be analysed statistically, for example using AI pattern recognition algorithms.






Figure 2: A sketch of 8 cubes being used to model a garden seat in an 1855 Kindergarten guide (left); a photo of a reconstruction of the sketched model with wooden cubes (centre); and a screenshot of a prototype 3D model generated from the reconstruction with photogrammetry app 3D Scanner (right). (The 3D model is viewable here: http://3d-viewer.xplorazzi.com/model-viewer/index.html?modelId=629e943a3aaf2b171525a9b5 )

With funding from the BDFI we were able to form a small project team of two researchers in the School of Education, and a software developer and the head of a local primary school, in order to develop an app to trial with children in the school.

Technical Challenges

The problem of capturing positions and orientations of blocks digitally almost immediately became more challenging than we had anticipated. Initially we had hypothesised that detection of straight edges would be a relatively simple computer vision task, however in practice traditional edge-detection algorithms proved unreliable in detecting the edges and extrapolating cubes positions, with multiple confounding issues including lighting, shadows, orientation, variations in perspective and vertical position, variations in wood texture and colour, and hidden edges under stacked blocks. One approach we attempted was to paint each block in a different colour to aid recognition, but this too was unsuccessful.

Figure 3. The move from plain wooden blocks to painted blocks to Cuisenaire rods to aid recognition

Finding ourselves stuck in terms of successful block recognition, we decided on two radical changes in direction: (a) to move from traditional edge-detection to AI-based computer vision algorithms, such as Mask-RCNN, and (b) to drastically simplify the recognition problem by focusing on Cuisenaire rods – standard classroom manipulatives which are 1 cm to 10 cm long, each coloured in a distinct colour, and typically arranged flat on the table, avoiding the issue of stacked blocks (Figure 3).

Our developer found that a gaming laptop equipped with a GPU processor was powerful enough to run Mask-RCNN, and with sufficient training on approximately 150 images, could detect the positions of Cuisenaire rods in an image from a live webcam feed within 2-3 seconds of processing time, which we felt was acceptable from a usability point of view.

With a feasible solution now successfully implemented for rod detection, the developer could now relatively easily add code which generated images and sounds associated with each rod, such as displaying a graphical image of it on screen, and speaking its colour or length. We trialled the app with Year 1 children in a local primary school, and produced a paper about the trial for the British Society for Research into Learning Mathematics.

Figure 4. The experimental set-up as used in the initial trial in a primary school

Lessons learnt

As educational researchers with little experience of developing apps such as this, we have learned many lessons. One is the value of iterative, so-called ‘Agile’ approaches which enable rapid experimentation and pivoting of direction in order to solve problems that inevitably arise in developing novel technologies.

Another is the value of the ecosystem of open-source libraries, shared expertise and documentation which grows over time around any novel technology, and in particular complex open-source AI algorithms and tools such as Google’s Tensorflow, and Facebook’s Detectron. Occasionally, a novel technology we tried looked attractive in terms of affordability – for example the OAK-D camera with built-in AI camera – but was so novel at the time that the supporting knowledge eco-system had not yet developed which effectively made it unfeasible to develop for in the short term.

And a third lesson learned is the critical importance of training data for AI computer vision algorithms/  or example, to recognise blocks placed on a school desk in daylight, the algorithm should be trained on images from as similar an environment as possible, but randomised sufficiently to avoid ‘overfitting’. This process of training AI algorithms also provided us with rich insights, from an educational conceptual perspective, into current neural network models and neuroscientific theories of how human brains learn – as well as some of the power and limitations of these theories.

Future challenges

With a prototype now delivered which can successfully recognise Cuisenaire rods, running on a GPU-equipped laptop and webcam, we are now looking towards potential future phases of development.. We’d like to revisit recognising plain cubes, and to make the app accessible on other devices like low-spec computers or mobile phones, allowing us to gather data on block play more widely from schools, as well as enabling children and their families to use the app at home.

We would also like to develop an AI app to analyse the block play data and recognise patterns, for example symmetries in constructions, or commonalities and differences across settings or over time, or compared with digital block play. Currently assessment of children’s activities in pre-school is often, like the curriculum, very different from primary school, and an app that could gather and showcase a portfolio of children’s real-world block play – potentially in virtual worlds if they wish – might enable more continuity in formative assessment across the transition from pre-school to primary.

Expanding the remit

We are also interested in the applications of a simple set of physical blocks as an interface, for example for playing musical notes, or modelling language, or atomic reactions in climate science, as well as for children with visual impairments who may not be able to see touch screens easily. And there also is the potential to translate the digital 3D models of children’s physical block constructions into current 3D online block metaverses such as Minecraft, to bridge the two worlds.

We are keen to work with partners across creative and technical disciplines who are interested in exploring opportunities to augment physical block play with multi-modal digital experiences. If you would be interested in learning more or a chat about the project please get in touch with us: alf.coles@bristol.ac.uk

Avon Street, Gas, and Bristol

We commissioned a report into the industrial and social histories of our new building at 65 Avon Street, known as The Sheds.  In the heart of the new Temple Quarter Enterprise Campus it was the former headquarters of the Bristol Gas Company.

Here one of the report authors, Dr James Watts, Lecturer in Public and Creative Histories describes how the project has unfolded and how its shed light Bristol’s industrial heritage. Co-author on the report,  Lena Ferriday, is continuing the research with a call for local people to come forward with their memories.

The Avon Street gasworks operated for nearly 150 years bringing light and heat to much of Bristol through the dangerous labour of those at the gasworks. Since April 2021 I have been researching this history for BDFI.

I was fortunate in beginning this project that research on the gasworks and their place in Bristol has been undertaken by others before me. Harold Nabb’s PhD thesis and pamphlet on the gasworks is invaluable as is Mike Richardson’s Men of Fire Work, Resistance and Organisation of Bristol Gasworkers in the Nineteenth Century alongside work by Michael Painting and Mike Richardson. Material on Know Your Place and in Bristol Archives has also been very helpful in digging into the history of the gas industry in Bristol in greater detail.

drawing of the gasworks by Samuel Loxton
Figure 1 The Avon Street gasworks, Samuel Loxton 1919, Bristol Library J785. By Permission of Bristol Libraries

The research revealed many links and parallels between the historical use of the site as a gasworks with the revolutionary effect this had on the life of Bristol.

The gasworks and the many local people employed there also had a profound effect on the local community, the workforce was locally drawn and, along with employers like the Ironworks across Silverthorne Lane, created a sense of community in this deeply industrial area of Bristol.


I was continually surprised by how far-reaching the technologies of gas heating and lighting were. Gaslighting created and extended the night-time economy, especially in the winter months. This meant that the centre of Bristol was lit from 1820 onwards. The Old Vic was an early customer of the gas company and remained one of their largest customers for many years and in 1869 was eligible for a special discount due to the volume that they were using which was more than 1 million cubic feet a year.

By the 1900s gas heating was also very common and pre-paid gas meters allowed tighter budgeting and enabled the spread of gas heating and cooking into working class households. There was a large showroom on Colston Street in the city centre, built in 1935, to advertise and sell gas cookers as well. This was demolished in 2007 and the building is now the Bristol Beacon.

gas showroom
Figure 2 Bristol Archives, Vaughan Postcard collection, 43207/35/1/2

The other thing that impressed itself on me was the sense of how much of a community the gasworks and the surrounding area seemed to be. The gasworks had football teams who were season champions in 1930-1 as well as a brass band. The solidarity of the workers in times of industrial action was remarkable as the gasworks were involved in the wave of strikes in Bristol in 1889.

gas workers football team
Figure 3 Bristol Gas Company Reserves Football Club, 5th Division Champions, Bristol and District League 1931, Bristol Archives, 28777/U/Ph/1/6

Hopes for the research and site

I think the Avon Street gasworks could act as an important example for the modern use of historic buildings. It is, for me, about respect. Respecting the building itself, but also an awareness of the people who made, used, worked, and lived in them. I hope that the buildings’ new uses will reflect this history and help to educate others about the history of this industry and area. Those stories should not disappear but should be considered and reflected upon in the future uses of the buildings.

For instance, George Daniel Jones was a gas holder attendant during the 1940s. On March 11th 1941 ‘during an air raid two incendiary bombs lodged on the top of a large gas holder. Jones immediately climbed to the top of the holder and succeeded in knocking the bombs off the crown with his steel hat.’[1]

For his bravery that day he was awarded the George Medal. There is also now a road named after George Jones as well as a plaque on Folly Rd on the site of a gasholder close to Avon St and also owned by the Bristol Gas Company.

My main hope for this research and the site is to find more stories and personal memories from the current outreach. What I want to know about the site are these personal stories of someone’s Grandfather who was a stoker or captained the gasworks’ football team.

It is personal stories that give the site its interest given the long history of work there.

We’ve created a short survey for anyone who might have memories, artifacts, documents  or photographs from the gas industry in Bristol.  Please get in touch to help us ensure the social and industrial heritage of BDFI’s new home is remembered and celebrated.


[1] Supplement to the London Gazette, 2 May, 1941. The recommendation is in the National Archives. https://discovery.nationalarchives.gov.uk/details/r/C14149725

Tackling an intelligence gap in 6G management and orchestration systems with HELICON

BDFI academic Xenofon Vasilakos, lecturer of AI for Digital Infrastructures with the Dept. of Electrical and Electronic Engineering and a member of Smart Internet Lab at the University of Bristol, discussed the current orchestration systems intelligence gap when devising 6G network services at the IEEE International Conference on Communications this week. Below, he explains a Reinforcement Learning model-based orchestration approach tested with Bristol’s 5G city testbed and a real use case, which tries to address this intelligence gap while aiming at multi-objective optimisation goals. Further, Xenofon explains how this work poses a basis for integrating and supporting sociotechnical aspects in the future, such as fair resource consumption by users and services.

Network softwarisation in the fifth and future sixth generation of wireless networks (5G, 6G) is characterised by significant flexibility and agility as a result of adopting the concepts of Software Defined Networking (SDN) and Network Function Virtualization (NFV). The former have enabled scalable vertical industry services with strict performance requirements that need to be addressed by MANagement and Orchestration (MANO) systems. Nonetheless, today’s state of the art in MANO systems faces fundamental challenges regarding the highly complex problem of optimal user service function placement. MANO systems still lack Machine Learning (ML) intelligence while remaining largely dependent on rule-/heuristic-based solutions focusing exclusively on system-level resources after predefined policies.

High-level HELICON architecture showing global RL (GRL) and local RL (LRL) modules (on top), and internal system component data and signal message exchange.

The above approach neglects critical technical aspects such as network dynamics and system-wide service-level performance objectives of both verticals and infrastructure providers as expressed by Key Performance Indices (KPIs) such as service latency or balanced resource utilisation. In addition, it neglects the potential of including Key Value Indicators (KVIs) such as user access fairness to deployed services to avoid user starvation.

To address these gaps, we propose and present our latest work entitled “HELICON: Orchestrating low-latent & load-balanced Virtual Network Functions” to the IEEE International Conference on Communications, in May 2022, Seoul, South Korea (https://icc2022.ieee-icc.org/), during the “QoE And Network Systems” leg of the technical symposium of “Next-Generation Networking and Internet (NGIN)”.

HELICON stands for “Hierarchical rEinforcement LearnIng approach for OrChestratiNg” low-latent and load-balanced services. Though targeting purely technical KPI-based objectives in the current stage, HELICON paves the way for introducing also KVI-based objectives into the MANO equation, thus setting the necessary technical background for supporting socio-technical aspects in current and future 6G MANO operations. In brief, HELICON:

  • Poses a novel distributed hierarchical Reinforcement Learning (RL) approach that can serve as a stand-alone online service placement solution as well as a module-based extension for the current state of the art.
  • Tackles a computationally/analytically difficult problem (NP-Hard) with a tunable and lightweight Q-Learning scheme that besides KPIs can also support KVIs in the future such as fair access to resources by both users and services. In its current pilot version, HELICON optimises either or both of (i) end-to-end service delay or (ii) load balancing among hosting service nodes.
  • Last, we provide a real-life testbed implementation and use case-driven validation, and specifically, practical experimental results upon a realistic 5G Smart City Safety (SCS) use case conducted over a Bristol’s 5G city testbed assuming an e2e application video transcoding service.

Choosing the ‘high road’: major employer study reveals remote working challenges and opportunities

Jennifer Johns at the School of Management has been working with a major UK employer during the last year to examine how their working practices have responded to COVID-19 challenges. What does blended working mean and how does this continue to impact on day to day business decisions?  Here she explains discoveries so far and implications for the world of human resources.

Within organisations and across media channels there is currently much discussion about the ways in which we work. Terms like ‘remote’, ‘hybrid’ and ‘blended’ learning are used to describe changing patterns of work, breaking the traditional assumption that we should work in an office location.  This is not a new trend.  Since the 1990s, increased use of communication technologies, particularly the Internet, has facilitated significant changes in the ways in which work is conducted.  Digital technology enables the multidimensional fragmentation of work – one form of fragmentation is spatial as work can take place across smaller and more isolated work units.  What IS new is the degree to which more flexible form of work are now taking place since the COVID global pandemic.

Before COVID, we saw a rise in the number of people working away from the office, typically from home.  This included full remote work (for example data processing, professional services) and part remote work (e.g. senior executives working from home two days a week). Academic work charted the rise of this work, but its increase was considered to be limited to a narrow range of job roles, predominately low skilled routine work that can be conducted online or, conversely, high skilled ‘white collar’ professional work.  We recently argued that existing academic understandings of remote work were overly simplistic and that the relationship between employees and employers could take a ‘high road’ in which employee wellbeing increases, or a ‘low road’ in which working conditions deteriorate over time.

During COVID, the national lockdowns introduced by national governments required organisations to make working from home mandatory for as many job roles as possible.  This meant questioning some old assumptions about what work had to be based in the office.  Many organisations realised that the move to paperless offices had decoupled some forms of work from the office (receptionists, salespeople now using electronic brochures etc). In some sectors, this left a relatively narrow number of job roles that were required to physically be present in the office, typically those involving the maintenance of critical business infrastructure.

Following the move of many employees to home working, organisations have had to respond with modified working practices, policies around the return to work and debates around how much flexibility to continue to offer employees when/if they return to work.  On one hand, organisations can make cost savings by reducing their office space. On the other, many are discussing what types of activities must be co-located, acknowledging that some employees want to return to the office, and working out which functions could remain at home.

Alongside collaborator Rory Donnelly (University of Liverpool), I have been working with a major UK employer  since April 2021 on their blended working practices. The initial introduction to this company was made by Bristol Digital Futures Institute. This employer will remain anonymous in the research findings, once published. We have interviewed over fifty employees across three different sectors, highlighting the different needs of individual divisions in relation to flexible work. This employer has much to share with other organisations about their ongoing experience of flexible working, particularly as their group ranges from customer-facing contact centres to maintaining critical infrastructure.  The notion of having contact centre agents working from home would have been inconceivable to many organisations pre-COVID (and many academics too).  Yet, their contact centre agents have been working from home effectively, generating higher customer feedback scores during COVID.  This has been incredibly illuminating about how organisations can support staff to work flexibly and how they can adapt to dramatic shifts in the business environment. Retail staff, who typically worked in physical shop locations, were trained to work from home as contact centre staff.  This demonstrates an agility not typically seen in large multinational companies.  Our findings are being fed back to the company via company-wide seminars and workshops.  Our work will continue with this company and extend to include others from different industry sectors. We will be generating wider impact through policy recommendations and industry briefings.

Challenges remain, as for most businesses, around how to embed flexible working within organisational cultures and how to maintain innovation and employee wellbeing with staff working in the office and from home.  Here the role of human resources professionals becomes especially important within organisations. So too is the role of academics in offering guidance on how businesses can achieve a ‘high road’ approach which values employee well-being and job satisfaction. These lessons will be valuable as we seek to understand now work might further change as a result of digitalisation.

Reflections on Soma

Angus Balbernie writes about his participation in Soma, a BDFI partnership project.


Soma combines somatic-based dance practices and immersive multi-person VR to explore pathways for sensory connections and examine how our interior and exterior worlds are mediated by technology. They are working towards premiering a new VR-dance film which reveals point of view experiences and creative audio descriptions of two dancers in a duet which crosses between physical and virtual bodies and environments at the Bloomsbury Theatre, London in May 2022.

soma dancers
credit: Alice Hendy for Soma

Today, as I’m writing, Storm Eunice blows hard here over Dartmoor, and

I’m watching the trees bend and break and thinking about being involved in Lisa May Thomas’s Soma Project.

I knew nothing about VR before being asked by Lisa to help mentor the project. I’ve never “gamed” in my life, never held two controllers and stared at activity on a screen.

Now I have. Now I’ve entered a world inside a headset. A world created and shared in a way JJ Gibson would have got all afferent and efferent about, in how we share our perceptions and relationships to the environment we are alive in. And yet I’m inside a plastic container, wrapped over my eyes, in a world generated from across the room – from inside a laptop.

I’m wrapped in a deserted world of blue sky, yellow dawn and shifting floor. I have no body, no relationship to the norms of place or navigation. I focus on my hands and the controllers as they affect this golden band floating around me. I can turn it and twist it and share it with others also in their VR state too. We have a form of exchange, and we begin to dance.

Something settles in this generated world. Something shifts in how the body senses itself as an activity that somehow responds to both an inner and outer sense of itself. What would normally be understood as anomic becomes shared. What is very isolating leaks into a shared environment.

There is something being shaped here, being danced, being created.

It happens in layers. The individual in the VR headset, and in a VR world.

The dancer who supports and informs them, surrounds them with their attention. The witness who watches and attends to both of them, a kind of satellite of attention, questioning and observing closely.

Then there are others at the edges of the space, watching all of this, seeing (or not seeing!) how the weaving of these various states, attentions and correspondences forms and dissolves in moments of order, disorder, dancing, choreography, theatre, science and perceptual exchange.

When the VR world is sharing its ribbons, which are moved through space by the collective hands of the participants, a dance emerges. A dance of delicate arms and bodies, held inside one space but translated into kinaesthetics to those observing. A dance of inner meaning translated into external space and action. A way to share someone’s dreams.

To share a world that generates the subjective, opens that out to a shared space, becomes objective, visible and invisible at different times, and then returns to asking how a generated state inside a computer can create a shared condition – of bodies being both alone and together in a world we usually only dream, or put “onstage” as an event?

At its heart, it asks questions about how, and why, art can still cut open our minds and bodies, turn us inside out, and realign our senses to be individually and collectively involved in what Ramachandram calls the “Figurative primitives of our perceptual grammar”.

At one point, towards the end of the VR journey, those inside the headsets see a growing constellation of generated dots and lines that form dancing webs of atomic structure. They are moved and condensed or opened out by the hands of those involved, and somehow it is a dance that takes us deep into how imaginations can be touched by something that is both seen and unseen, and how dancing is also found in the darkness, and in the ways we try to pass our realities to each other.

We are dancing ourselves inside out. We are asking the body questions.

We are making sense of each moment, both sharing and locked in self.

We are becoming what is now becoming.

It all just dances around you anyway.

To unlock the nervous system and make it a form of reading.

To dance around fact like it is a verb.

To shape invention through body and mind to make sense

of how time makes space a navigation, a way to shape experience

in what we think is isolation, into a form of making mind and body both alone and shared, imagined and earthly, and most of all, still there.

The blog first appeared on the Soma blogspot in March 2022