Evaluation

Overview | Concept | DIYbio | Workshops | Exhibition | Evaluation | Conclusion

Apart from the scientific result – A Public BioBrick – we also wanted to find out what participants learned from the collaboration, how it has helped the community forwards and how in-depth the collaboration was.

Depth of Collaboration

One of the important evaluation criteria for citizen science projects is the “degree of participation”, ie. how involved the citizen science collaborators were at different stages of the project. We designed a simple survey (adapted from "Public Participation in Scientific Research: a Framework for Deliberate Design", see reference) that each collaborator filled in after the workshops.

The results are a validation of our approach: both iGEMers and Biohackers felt they contributed in equal parts towards the collaboration.

Interviews of Workshop Participants

We held semi-structured interviews of about 30 minutes with 5 iGEM students and 4 participating Biohackers, to evaluate the results of the workshop.

iGEM Participants

What’s unique about the Biohackers?

“The Biohackers come from different backgrounds, but they’re united by the idea of doing biology, exploring it in its different aspects.”
“They’re very keen to know and understand each step in a protocol and why that step is important. They want to get the science behind it and sometimes go even deeper than us student scientists, because for us, it’s often just a procedure, but for them, everything is interesting.”
“Their approach to science leads them to really see each step in its own right.”
“I was very surprised to find a group of people who are so motivated and curious about biology that they’re willing to give up their spare time for this project. They’re very dedicated.”

What are the Biohackers’ motivations?

“Pure desire to discover more about the world around them. It’s a lot closer to the renaissance spirit of exploration than what we normally see around us.”
“I think they’re interested in synbio because of the resemblance to electronics, which is the background for a lot of them.”

How do you think the Biohackers will develop?

“Science is continuously trying and re-trying with different conditions. So the fact that the Biohackers haven't got so many results before at the Hackspace is not as demotivating for them now that they see that [academic science] is not always successful either.”
“I think the Biohackers gained a lot of experience in terms of structure because within science, the steps to achieving a specific goal can sometimes be very hazy.”

What did you learn about supervising amateur scientists?

“I learned that enthusiasm and excitement for the field counts for a lot. Even though the Biohackers didn't have training, because of their interest they pick up stuff quickly.”
“Sometimes we're skipping too many details. [When working with amateur scientists] you need to think more with them about what you're doing. I actually really enjoyed that; it made me question every ingredient of the experiment.”

Could the collaboration be improved next time?

“The collaboration was fairly unidirectional in terms of we were supporting them, and less of other way around. There are many projects, like creating new devices, in which they could have been a great help to our project.”
“Targeting a more diverse range of biohackers.”

What’s the difference between the Biohackers / amateur scientists and professional academics (other than the trivial: “they’re getting paid for it”)

“We work within the established infrastructure of academia; I think it's easier for the biohackers to think out of the box when it comes to looking at problems that can be solved by synthetic biology or biohacking.”
“Academic build their knowledge step by step, but a biohacker may not have that structure of knowledge - they have gaps here and there, so their knowledge isn't so well organized.”

Community-level Outcome

“It can bring a community together, people that wouldn't normally meet, by fostering a sense of common interest.”

How would you describe your overall experience?

“I really enjoyed it. I wish I had spent more time doing it because it was learning experience for me as well as for them.”
“We were mutual teachers; [he] asked lots of questions on biology, but I asked him lots about electronics.”
“It was eye opening!”

Biohackers

What new skills / knowledge did you gain?

“Specifically, equipment hygiene.”
“It’s not just about curiosity for scientists, they also need to be disciplined .”
“I found the experience of [the workshop experiments] not working very positive because that happens [in the Hackspace] a lot and it's nice to know that also happens with other people. If it goes right, you just followed procedure but didn't gain understanding but with troubleshooting you learn… so we got a lot out of that exercise.”

Which existing skills did you develop further?

“Interpreting… using controls, positive, negative controls on tests, so you know what you did wrong.”

What did you learn about working in an academic lab?

“One of the instructors said, when students do [similar experiments] it takes a month to get it done, but because we were biohackers and wanted to do this, learning stuff was faster, because we were all passionate and learned faster.”
“Some of the striking differences that I come across are how [academia] funds stuff and how you decide to do it.”
“There is more division of labor in academic lab, they have technicians and buy equipment, etc. while we have to make equipment, clean ourselves, etc.”

Describe the workshop program. Which experiments did you undertake?

“We started at Hackspace firstly, trying to make competent E. coli as a carrier for the biobrick. Then we amplified the plasmid backbone, which would be to carry the biobrick. Then we went to UCL and amplified the genes from Oceanibulbus indoloflex - the antifreeze and mercury reductase - and now we're back at the Hackspace trying to amplify these genes again in our own environment.”

What was your motivation to participate?

“We needed some outside input to improve our techniques and experiments.”
“I was interested in getting a grasp on the professional techniques, taking [my skills] to the next level.”

What specific aims did you have for the workshop?

“I wanted to raise the quality of our work in the backspace.”
“One of my aims was to collaborate and meet other great people.”

Could the workshop have been improved?

“It would be great to have some learning & illustrative videos from the whole process… maybe we can do that in the Hackspace.”

Did the workshop add to your perspective on Biohacking?

“[Our collaboration] gave [biohacking] a more professional perspective”
“I don't see a need to separate the collaboration and biohacking. We're doing the same sort of stuff, the difference is the social context that we're doing it in. If I were an academic working on a lab, that would be my living and that would put constraints on it, because I’d have deadlines to keep and people assess my work. The interesting thing about [biohacking] is that it is less constrained in that way but we created other kinds of constraints for us in the process, particularly resources.”

References

Shirk, J. L., H. L. Ballard, C. C. Wilderman, T. Phillips, A. Wiggins, R. Jordan, E. McCallie, M. Minarchek, B. V. Lewenstein, M. E. Krasny, and R. Bonney. 2012. Public participation in scientific research: a framework for deliberate design. Ecology and Society 17(2): 29.