From 2012.igem.org

(Difference between revisions)

Latest revision as of 20:00, 26 September 2012

Scientific impact of iGEM

"Most influential synthetic biology competition" vs. "Just some kids playing"?

We wanted to determine how relevant the iGEM competition is for the greater SynBio community. So we investigated the scientific attention garnered by both the iGEM and the Registry of Standard Parts. A data-driven approach was chosen: We extracted data from search results using various queries (such as ("iGEM" OR "International Genetically Engineered Machine") AND ("synthetic biology" OR "genetic engineering")) from various publication search engines. We searched Web of Knowledge, Scopus, PubMed and Google Scholar. Google Scholar was chosen to perform more detailed data analysis, as we found the alternatives to have various shortcomings.

We found that our data results are conclusive with our initial hypothesis: iGEM is an important contributor to the SynBio community. These findings have some implications for the iGEM competition, which we discuss.

In order to quantify these results further, we analyzed how exactly iGEM and the Registry has been cited. Examining around 50 papers in closer detail, we recommend all papers published by iGEM teams or related to iGEM or the Registry to use a standard citation.

Query summary

Here's a quick breakdown of what we queried for on Google Scholar and what sort of data was returned. (The ID matches the name of the data set in our data tables). The h- and g-indexes are explained just below!

Dataset ID	Plain English query	Query	Nº Papers	Nº Citations	h-index	g-index	Query date
1	Papers mentioning iGEM in context of synbio	`(“synthetic biology” OR "genetic engineering") AND (“iGEM” OR “International Genetically Engineered Machine")`	770	3253	26	45	17/7/2012
2	All synthetic biology	`synthetic biology`	1000	68482	127	214	17/7/2012
3	Papers mentioning Registry of Parts	`"Registry of Standard Biological Parts" OR "partsregistry.org" OR "parts.mit.edu"`	751	6442	39	69	17/7/2012
4	Papers citing a particular Part	`partsregistry.org/Part:`	54	263	5	16	17/7/2012
5	Papers mentioning iGEM	`iGEM OR "International Genetically Engineered Machine"`	1000	9095	36	64	17/7/2012
6	Papers mentioning iGEM and Registry	`("iGEM" OR "International Genetically Engineered Machine") AND("Registry of Standard Biological Parts" OR "partsregistry.org" OR "parts.mit.edu")`	330	2208	23	42	17/7/2012

Note Searches were capped at a maximum of 1000 results. Hence getting 1000 results for a query implies that more exist! Those first 1000 are only the ones the search engine judged most relevant.

Why we used Google Scholar

All in all, we found Google Scholar to most closely meet our analytic needs.

As WoK, Scopus and PubMed are strictly curated databases and limited in scope, they missed many obviously relevant publications. We also found their search options unsuitable: Many of them did not support either full text search (they looked at titles, keywords and abstracts only) or boolean operators. But for the following reasons, we needed a search to include both:

iGEM cannot be expected to always be the main subject of a paper, hence full text search.
There are many relevant terms floating about iGEM, hence boolean operators like "OR" to allow treating papers that contain "International Genetically Engineered Machine" or the acronym "iGEM" equally.

Just how much wider is Google's search scope? Here is an example: PubMed gave so few results (16 for iGEM genetic*) that we quickly discarded it. Manually merging the Web of Knowledge and Scopus results for the query iGEM AND genetic* (discarding obviously irrelevant results) gave 43 results. Then we queried Google Scholar. It gave us 770 for (“synthetic biology” OR "genetic engineering") AND (“iGEM” OR “International Genetically Engineered Machine").

Of course, Google Scholar too is but a bronze bullet: It brings its own drawbacks. It is engineered to pick up things that only seem like scholarly articles. Like Google's search results in general, the results are not curated by a human. This has been criticised in the literature (Péter J., 2006.). We found the occasional hilarious total miss. Google Scholar is also known to somewhat overestimate citation counts (Iselid, L., 2006.). However, we concluded from empirical manual examination of a random sample that the majority of the results are plausible and (most importantly) far greater in scope than searches in curated databases. Taking these aspects into consideration, we found Google Scholar best fulfilled our requirements.

Caveat! We only wanted to identify trends. Large and coarse pieces of data with some error were sufficient for this. These values should not be taken as exact!

Browse the data

...are online in a nifty and very usable Google Docs folder.

An introduction is included in case you get lost or want more information.

On extraction tools

We made extensive use of Harzing Publish or Perish (Harzing, A.W., 2007.) to scrape Google Scholar results. The tool has many limitations. However, in our experience it is the best available resource for managing the many locations in which scientific publications are deposited.

We did try other things: Manual methods were too slow for hundreds of papers. Various Firefox browser plugins failed due to the search APIs having changed, were extremely awkward to use or produced clearly erroneous results. The Mac OS program Papers was easy to use and found large numbers of papers (as like Google Scholar it could access many sources), but had unacceptably high rates of error, problems with duplicates and could not export the results into a form we could easily process. Hence Publish or Perish.

Once we had scraped our data from Google Scholar, we needed a method to quantify the relevance of a given scientific article. There are many ways you can quantify success of a paper. Here are a few we investigated:

Plain citation count

High citation count can generally be taken as an indicator of a high-quality or high-impact paper. This is the most traditional method of ranking the influence of papers.

The main disadvantage of the "citation count"-method is its lack of consistent standards. Even papers within a distinct scientific fields will have differing reference approaches and citation counts. It is also significant that old papers have an edge over newer papers, as they've had more time to be cited.

h-index

The h-index is an integer unique to a set of papers. It is used to measure the output and influence of a set of scientists. A greater h-index implies more productive and more influential authors. It was invented by physicist J.E. Hirsch (2005) and has since been automatically calculated by many citation databases. Here is its definition: "A set of papers has h-index h if h papers out of that set have been cited at least h times." An image ("Ael 2" and "Vulpecula", 2012.) clarifies:

g-index

The g-index is a citation index meant to quantify the influence of papers. It was proposed by Leo Egghe (2006) as a variation to the h-index. It puts more emphasis on the most cited papers and Egghe argues that it ranks highly cited authors more fairly. He gives the following definition: "A set of papers has a g-index g if g is the highest rank such that the top g papers have, together, at least g² citations." Here's a clarifying image by our Polish friend ("Ael 2", 2012.) again:

Algorithmic methods

It's worth noting that there are many other ways of quantifying productivity and impact of a set of papers or scientists. For example, Y.B. Zhou et al (2012) propose a more complete method for "distinguishing prestige from popularity". In their algorithm, the weight of a citation to the influence of a paper is also dependent on the (already calculated) influences of the citing papers and their authors. This requires running a recursive algorithm on sufficiently complete bipartite network of papers and their authors.

Though the alternatives look enticing, we ended up looking mainly at citation count.

The algorithmic methods were beyond our reach of data availability: We would have had to find the names of every involved person in every iGEM team and all papers they've written, filtering out large amounts of false matches. This was impracticable.

The h and g-indexes don't actually show more than the raw citation counts when it comes to tendencies over time. Also, we had relatively few search queries to compare against each other, given that two were discarded for having various flaws (discussed just below). This meant that the h and g-index, while valuable methods, were not suitable for our particular data analysis.

Is our data usable?

Yes. Mostly...

Given our doubts over accuracy of Google Scholar's data, we considered it a priority to exercise caution with our search query results. This paid off: data we compiled using the search queries of IDs 5 and 2 had fatal flaws. They were rejected from further analysis (discussed below). The other data sets were found to be suitable.

Our method to examine data suitability was empirical: Random samplings of each data set were passed under human eyes. For most queries, this observation of random subsets showed acceptably low levels of "background static" (i.e. results that Google Scholar had automatically matched to the query, which were not actually relevant). These would form only a drop of error in the ocean of relevant data.

...but only mostly.

Query 5 (iGEM OR "International Genetically Engineered Machine") was found to have an unacceptably large level of static. The reason was quickly identified: Because the two quoted terms in Query ID 5 ("iGEM", "International Genetically Engineered Machine") were separated by a disjunction (OR), the query would easily match anything that contained just the acronym "IGEM"! This meant acronyms in economics such as "Inter-temporal General Equilibrium Model (IGEM)" or the British "Institution of Gas Engineers & Managers (IGEM)" and various medical terms and chemical names snuck in. The entire data set with ID 5 was dismissed from further analysis.

The lesson we took from this is not to search for short acronyms by themselves. Query 1 ((“synthetic biology” OR "genetic engineering") AND (“iGEM” OR “International Genetically Engineered Machine")) could be thought of as the "Version Two" of the problematic Query 5. It searches for the same terms, but includes a conjunction (AND) with either synthetic biology or genetic engineering, which bends results toward our iGEM. This tunes static down to an acceptable level.

Query 2 (synthetic biology) had a different problem: It was too big. The query was an attempt to capture stats for the entire field of synthetic biology, so we could statistically determine the relative influence of the iGEM competition. However, we had forgotten the 1000-result cap imposed by Google Scholar. It is impossible to retrieve results beyond this 1k "event horizon". Google does not publish information regarding how the order of results is determined. Hence these first 1000 results (out of what are likely to be 10s or 100s of thousands of papers) are all biased by some unknown force. Were more cited papers favoured? Were papers published more recently favoured? No conclusions can be drawn from a biased and small subset of the full data. We also discounted data set 2 from any further analysis.

To emphasize: Data sets 2 and 5 are not included in any further analysis.

What's is iGEM's impact?

What does "impact" mean within a scientific context? We will operate under the assumption that having impact correlates strongly with being mentioned in scholarly articles. Hence, we quantify the impact that a term is having simply by searching for that term, and summing up result counts.

Here is a chart summarising how many papers mention various terms floating about iGEM over time:

Number of teams Parts submitted (10s) Papers mentioning iGEM Papers mentioning Registry of Parts Papers mentioning specific Registry Parts

Chart 1: Summary of iGEM-related attention over time

This chart shows the amount of "attention" the iGEM competition is receiving over time, shown relative to the growth of the competition itself (quantified by amount of registered teams). "Attention" is measured by amount of papers mentioning certain keywords deemed to be relevant to iGEM or the Registry of Standard Parts. An overall positive trend in proportion with the growth of the competition can be seen.

And the answer is...

Things are looking good for iGEM! Since the first iGEM competition in 2003, more teams are participating each year and more parts are being submitted. The efforts to expose iGEM to the community has paid off - the amount of scientific papers mentioning iGEM and the Registry of Standard Parts has risen, proportional to the increase of the competition itself.

Papers mentioning a specific Registry BioBrick have only begun to appear in recent years, but the numbers show growth. We hypothesize that the Registry's contents is only now reaching the critical mass to become a useful research tool. The founders' dream (iGEM Foundation, 2012b) of "making genetics modular" is becoming reality.

Where this data comes from

Data about participating teams and number of submitted BioBricks comes from the iGEM Foundation (2012a). The other data sets come from the results Google Scholar queries with IDs 1, 3 and 4 (see query summary table above).

Is the relationship between iGEM and the Registry clear?

Papers published in that year Times those papers were cited

Chart 2: "iGEM" attention
Papers published in that year Times those papers were cited

Chart 3: "Registry" attention
Papers published in that year Times those papers were cited

Chart 4: "iGEM and Registry" attention

These three graphs show the amount of papers published each year containing certain search queries, as well as the number of times these papers were cited. All graphs show positive tendencies: the competition is becoming more wide-spread and more iGEM-related papers are being published and recognized. The search queries were chosen to show which part of iGEM is usually cited: the iGEM competition, the Registry of Standard Parts, or both. The data shows that only around half of papers will cite both elements. Some far outlying data points (clear errors) were not plotted to maintain uniform scale.

Expectations

One of the iGEM competition's important goals is to build up well-characterized BioBrick content in the Registry. Thus, the iGEM competition and the Registry are inherently linked. Hence we would expect publications that mention iGEM to also refer to the Registry. However, as the Registry is not only used by iGEM, we expected to also find a large number of papers that mention the Registry without mentioning iGEM.

In other words, we expected:

a large proportion of papers that mention iGEM and the Registry,
a large proportion that mention only the Registry and
only a very small proportion that mention only iGEM.

Yes and No

The expected proportion of papers mentioning only the Registry was found. However, of the papers that mention iGEM, about half do not mention the Registry. This was a greater proportion than we expected.

We theorise that this may be due to the iGEM Foundation and iGEM teams under-emphasizing their importance to part standardisation. Indeed iGEM is usually presented by the Foundation and teams as "a synthetic biology competition", when really that's just half the picture. We're not just competing, but we also exist for a greater good: Make the Registry better and do our part in helping organise synthetic biology!

Just iGEM iGEM and Parts Registry Just Parts Registry

Chart 5: Proportional mentions of iGEM, Registry and Both

This chart is a stacked combination of the three charts above. It shows how often iGEM, the Registry and both are mentioned in scientific articles (i.e. how many papers from each year match our search terms). Each of the three quantities has stayed roughly equal to the others throughout the years.

Sourcing data

In order to analyse how exactly iGEM and the Registry are being cited, we decided to manually examine a set of the papers in our results. We had discarded Scopus and Web of Knowledge earlier when carrying out wide-range data collection. However, for this focused search, the small but certain selection of papers that Scopus and Web of Knowledge gave us was perfect!

We manually combined the results obtained from Scopus and Web of Knowledge. We then deleted duplicates and the few remaining irrelevant publications.

The keyword “iGEM” gave 41 publications combined from Scopus and WoK. Of these, we discarded 16 publications for various reasons:

5 texts were non-research articles, such as magazine articles
2 were articles on bioethics that mentioned iGEM only in passing
1 article cited an articles with “iGEM” in the title but was itself unrelated
6 were articles we couldn’t locate or access despite the University of St Andrews having subscriptions to various publishers
2 were in French.

This left 25 articles.

Discussion

We considered citations sufficient when they cited the Registry or named the specific BioBricks used (when appropriate) and citing either the Knight or Endy paper about BioBrick assembly.

We examined those 25 articles for how they cited iGEM and the Registry. We found that 11 articles cited iGEM and/or the Registry of Standard Biological Parts satisfactorily.

There were two common forms of citation:

Registry of Standard Biological Parts [http://www.partsregistry.org].
Knight, T. F. (2003). Idempotent Vector Design for Standard Assembly of Biobricks.
DOI: 1721.1/21168.

5 articles did not cite the Registry or BioBricks sufficiently according to our demands. For one of them, we question its content relevancy. The remaining 4 show examples of simple in-text citation of the Registry, with or without a hyperlink, no mention of the specific BioBrick used, or no mention of BioBricks at all. Surprisingly, each of these papers was associated with an iGEM team.

There are 9 additional papers we found in the journal IELTS Synthetic Biology, all of which show vast variations in content, research, and citation quality. The journal seems to have been published once, and seems to have asked all 2006 iGEM finalists to submit a paper based on their research. This resulted in some sub-par articles. Because they weren't peer-reviewed and due to their strong biasing affiliation with iGEM, we disregarded them from our overall data set.

Our recommendations

Reviewing the data, we have concluded that there are standard methods of citation being used by the scientific community to refer to the Parts Registry. In order for the Registry to uphold referencing standards as well as Parts standards within synthetic biology, we think this method of referencing should be officially recommended on the Registry and iGEM website.

A clear and standard citation method would support teams who are attempting to publish and let them set an example of citation style to the rest of the scientific community. Additionally, clearly stating a standard method of citation would make citing of the Registry easier and so motivate its citation in general. Of course, more citations means more attention and adoption within the field of synthetic biology.

Teams participating in iGEM should be encouraged to cite properly and to try to publish their work. A tutorial of some kind hosted on the iGEM website would help. Such things (among many other bright ideas) were proposed in 2008 by Cowell, but haven't been implemented. We see a standardised citation method as a high priority for maximising iGEM's scientific influence.

We found that the iGEM competition is making a positive impact. The competition is growing in size and scope, and both iGEM and the Registry are netting a proportionally increasing amount of attention from the scientific community. We are doing well!

However, we also found that quite a number of discussions of iGEM miss the important connection between our iGEM competition and the Parts Registry. We recommend that the iGEM Foundation and future teams emphasise the iGEM competition's raison d'être clearly in the future.

We also noticed that some papers do not give sufficient or clear credit to the Registry. We interpret this as confusion as to how the Registry should be cited. We recommend that a standardised referencing should be introduced. This would support the publishing process for inexperienced undergraduates involved in the competition, gather greater attention for the iGEM Foundation and Parts Registry and hence further our aims of synthetic biology standardisation. ∎

"Ael 2" and "Vulpecula", 2012. h-index (Hirsch). Wikipedia. [image online] Available at: <http://en.wikipedia.org/wiki/File:H-index-en.svg> [Accessed Jul 27, 2012].

"Ael 2", 2012. Illustrated example for the g-index proposed by Egghe. Wikipedia [image online] Available at: <http://en.wikipedia.org/wiki/File:Gindex1.jpg> [Accessed Jul 27, 2012].

Cowell, M.L., 2008. Making iGEM Better. [web page] Available at <http://openwetware.org/wiki/User:Macowell/Making_iGEM_Better> [Accessed Jul 19, 2012].

Egghe, L., 2006. Theory and practise of the g-index. Scientometrics [online], Volume 69 (Issue 1), p.131-152. Available at: <www.springerlink.com/content/4119257t25h0852w/?MUD=MP> [Accessed Jun 7, 2012].

Harzing, A.W., 2007. Publish or Perish. [computer program] Available from <http://www.harzing.com/pop.htm>

Hirsch, J.E., 2005. An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences of the United States of America, Volume 102 (Issue 46). [online] Available at: http://<www.ncbi.nlm.nih.gov/pmc/articles/PMC1283832/?tool=pmcentrez> [Accessed 5th Jul, 2012]

iGEM Foundation, 2012a. Previous iGEM Competitions. [web page] Available at: <https://igem.org/Previous_iGEM_Competitions> [Accessed Jul 30, 2012]

iGEM Foundation, 2012b. Press Kit. [web page] Available at: <https://igem.org/Press_Kit> [Accessed Aug 3, 2012]

Iselid, L., 2006. Research on citation search in Web of Science, Scopus and Google Scholar. One Entry to Research [blog] Available at: <http://oneentry.wordpress.com/2006/08/11/research-on-citation-search-in-web-of-science-scopus-and-google-scholar/> [Accessed Jun 20, 2012].

Péter J., 2006. Dubious hit counts and cuckoo's eggs. Online Information Review [online] Volume 30 (Issue 2) p.188-193. Available at: <http://www.emeraldinsight.com/journals.htm?articleid=1550726&show=abstract> [Accessed Jun 20, 2012].

Zhou Y., Liyan L. and Menghui L., 2012. Quantifying the influence of scientists and their publications: distinguishing between prestige and popularity. New Journal of Physics, [online] Volume 14 (March 2012) Available at: <http://iopscience.iop.org/1367-2630/14/3/033033/> [Accessed Jun 7, 2012].

Team:St Andrews/Human-practices

From 2012.igem.org

Latest revision as of 20:00, 26 September 2012

Scientific impact of iGEM

Introduction

Data collection Data and where to find it

Query summary

Why we used Google Scholar

Browse the data

On extraction tools

Metrics Quantifying scientific impact

Plain citation count

h-index

g-index

Algorithmic methods

Data analysis What does this data actually mean?

Is our data usable?

Yes. Mostly...

...but only mostly.

What's is iGEM's impact?

Chart 1: Summary of iGEM-related attention over time

And the answer is...

Where this data comes from

Is the relationship between iGEM and the Registry clear?

Chart 2: "iGEM" attention

Chart 3: "Registry" attention

Chart 4: "iGEM and Registry" attention

Expectations

Yes and No

Chart 5: Proportional mentions of iGEM, Registry and Both

Citations How is iGEM cited and what can we learn from it?

Sourcing data

Discussion

Our recommendations

Conclusion Our Human Practices in a nutshell...

References

@@ Line 3: / Line 3: @@
 <html lang="en">
 <head>
-<link rel="stylesheet" href="https://2012.igem.org/Team:St_Andrews/Template:DocsCssFull?action=raw&ctype=text/css" type="text/css" />
+<style type="text/css">
-<script type="text/javascript" src="https://www.google.com/jsapi"></script>
+.label-set .label {line-height:1.8}
-    <script type="text/javascript">
+#content-container .page-header {padding-top:30px}
-      google.load("visualization", "1", {packages:["corechart"]});
+.extra-spacing {padding-top:20px}
-      google.setOnLoadCallback(drawChart);
+.thumbnail {text-decoration:none !important;background-color:#ffffff}
-      function drawChart() {
+.thumbnail:hover {color:inherit !important}
-        var data = google.visualization.arrayToDataTable([
+</style>
-          ['Year', 'Biobricks'],
-          ['2004',50],
-['2005',125],
-['2006',724],
-['2007',800],
-['2008',1387],
-['2009',1348],
-['2010',1863],
-        ]);
-        var options = {
-          title: 'Company Performance'
-        };
-        var chart = new google.visualization.LineChart(document.getElementById('chart_div'));
-        chart.draw(data, options);
-      }
-    </script>
-<script type="text/javascript">
-      google.load("visualization", "1", {packages:["corechart"]});
-      google.setOnLoadCallback(drawChart);
-      function drawChart() {
-        var data = google.visualization.arrayToDataTable([
-          ['Year', 'Teams'],
-          ['2004',5],
-['2005',13],
-['2006',32],
-['2007',54],
-['2008',84],
-['2009',112],
-['2010',130],
-        ]);
-        var options = {
-          title: 'Company Performance'
-        };
-        var chart = new google.visualization.LineChart(document.getElementById('chart_div2'));
-        chart.draw(data, options);
-      }
-    </script>
 </head>
    <body data-spy="scroll" data-target=".subnav" data-offset="50" id="body-pattern">
+     <div class="container" id="content-container" style="background-image:url('https://static.igem.org/mediawiki/2012/a/ac/ImpactLogo_backgroundCorner_plusTransp_right.jpg');background-repeat:no-repeat;background-position:top right;background-size:35%">
-     <div class="container" id="content-container">
 <!-- Masthead
@@ Line 63: / Line 20: @@
 <header class="jumbotron subhead" id="overview">
    <h1>Scientific impact of iGEM</h1>
-   <p class="lead">"<em>most influential synthetic biology competition</em>" vs. "<em>kids' playground</em>"</p>
+   <p class="lead">"Most influential synthetic biology competition" vs. "Just some kids playing"?</p>
    <div class="subnav">
      <ul class="nav nav-pills">
        <li><a href="#introduction">Introduction</a></li>
        <li><a href="#data-collection">Data collection</a></li>
+      <li><a href="#metrics">Metrics</a></li>
        <li><a href="#data-analysis">Data analysis</a></li>
+      <li><a href="#citations">Citations</a></li>
+      <li><a href="#conclusion">Conclusion</a></li>
+      <li><a href="#references">References</a></li>
      </ul>
    </div>
@@ Line 78: / Line 39: @@
 <!-- Introduction
 ================================================== -->
-<section id="introduction">
-	<br>
-	<h1>Introduction</h1>
-	<p>We investigated the scientific attention garnered by iGEM and the Registry of Standard Parts. A data-driven approach was chosen: We extracted data from search results to various queries (such as <code>("iGEM" OR "International Genetically Engineered Machine") AND ("synthetic biology" OR "genetic engineering")</code>) from various publication search engines. We searched <a href="http://apps.webofknowledge.com/">Web of Knowledge</a>, <a href="http://www.scopus.com/home.url">Scopus</a>, <a href="http://www.ncbi.nlm.nih.gov/pubmed/">PubMed</a> and <a href="http://scholar.google.co.uk/">Google Scholar</a>. Google Scholar was chosen due to the alternatives' various shortcomings.</p>
+<section id="introduction" class="span9">
-	<p>We conclude with hypotheses to explain the results and discuss their implications for the iGEM competition.</p>
+<div class="page-header">
-	<p><span style="background:#ffff00"> TODO: Findings summary here too </span></p>
+	<h1>Introduction</h1>
+</div>
+<img src="https://static.igem.org/mediawiki/2012/a/ac/ImpactLogo_100.png" align="left"></img>
+	<p>We wanted to determine how relevant the iGEM competition is for the greater SynBio community. So we investigated the scientific attention garnered by both the iGEM and the Registry of Standard Parts. A data-driven approach was chosen: We extracted data from search results using various queries (such as <code>("iGEM" OR "International Genetically Engineered Machine") AND ("synthetic biology" OR "genetic engineering")</code>) from various publication search engines. We searched <a href="http://apps.webofknowledge.com/">Web of Knowledge</a>, <a href="http://www.scopus.com/home.url">Scopus</a>, <a href="http://www.ncbi.nlm.nih.gov/pubmed/">PubMed</a> and <a href="http://scholar.google.co.uk/">Google Scholar</a>. Google Scholar was chosen to perform more detailed data analysis, as we found the alternatives to have various shortcomings.</p>
+	<p>We found that our data results are conclusive with our initial hypothesis: iGEM is an important contributor to the SynBio community. These findings have some implications for the iGEM competition, which we discuss.</p>
+<p>In order to quantify these results further, we analyzed how exactly iGEM and the Registry has been cited. Examining around 50 papers in closer detail, we recommend all papers published by iGEM teams or related to iGEM or the Registry to use a standard citation.</p>
 </section>
@@ Line 90: / Line 53: @@
 ================================================== -->
 <section id="data-collection">
-	<br>
+<div class="row">
-	<h1>Data collection</h1>
+<div class="span12">
-	<h2>Why Google Scholar? (Why not Web of Knowledge, Scopus, PubMed?)</h2>
+<div class="page-header">
-	<p>As iGEM is somewhat of a niche topic and WoK, Scopus and PubMed are strictly curated and limited in scope, they missed many obviously relevant publications. We also found their search options unsuitable: They often did not support full text search (they looked only at titles, keywords and abstracts) or boolean operators. These were a requirement: iGEM cannot be expected to always be the main subject of a paper, hence full text search. There are many relevant terms floating about iGEM, hence boolean operators like "OR" to allow treating papers that contain "International Genetically Engineered Machine" or the acronym "iGEM" equally. Google Scholar had none of these drawbacks.</p>
+	<h1>Data collection <small>Data and where to find it</small></h1>
-	<p>What sort of scope range are we talking about here? This range: PubMed gave so few results (<strong>16</strong> for <code>iGEM genetic*</code>) that we quickly discarded it. Manually merging the Web of Knowledge and Scopus results for the query <code>iGEM AND genetic*</code> (discarding irrelevant results) gave <strong>43</strong> results. Then we queried Google Scholar. It gave us <strong>770</strong> for <code>(“synthetic biology” OR "genetic engineering") AND (“iGEM” OR “International Genetically Engineered Machine")</code>. Imagine our faces.</p>
+</div>
-	<p>Of course, Google Scholar too is but a bronze bullet: It brings its own drawbacks. It is engineered to pick up things that only <em>seem like</em> scholarly articles. Like Google's search results in general, the results are not curated by a human. This <a href="http://www.emeraldinsight.com/journals.htm?articleid=1550726&show=abstract">has been criticised in the literature</a>. Also we found the occasional <a href="https://2012.igem.org/File:TokyoTechNotReallyNature.png">hilarious total miss</a>. Google Scholar is also <a href="http://oneentry.wordpress.com/2006/08/11/research-on-citation-search-in-web-of-science-scopus-and-google-scholar/">known to somewhat overestimate citation counts</a>. However, we concluded from empirical manual examination of a random sample that the majority of the results are plausible and (most importantly) <em>far</em> greater in scope than searches in curated databases. We want to identify trends and for that, large and coarce pieces suffice.</p>
+	<h2 id="query-summary">Query summary</h2>
+	<p>Here's a quick breakdown of what we queried for on Google Scholar and what sort of data was returned. (The <em>ID</em> matches the name of the data set in our <a href="https://docs.google.com/folder/d/0B6xW2A1PIytESjJ4SUlteG9rUEU/edit?pli=1">data tables</a>). The <i>h-</i> and <i>g-indexes</i> are explained just below!</p>
-	<h2>Summary of queries we ran</h2>
-	<p>Here's a quick breakdown of what we queried for on Google Scholar and what sort of data was returned.</p>
 <table class="table table-striped">
 <tr>
+<th>Dataset ID</th>
 <th>Plain English query</th>
 <th>Query</th>
@@ Line 112: / Line 74: @@
 <tr>
-<td>Papers mentioning iGEM</td>
+<td><a href="https://docs.google.com/folder/d/0B6xW2A1PIytESjJ4SUlteG9rUEU/edit?pli=1&docId=0AqxW2A1PIytEdDBNV2ExQnluOXJ6SzlqeEZSQ2twc1E">1</a></td>
-<td><code>iGEM OR "International Genetically Engineered Machine"</code></td>
-<td>1000</td>
-<td>9095</td>
-<td>36</td>
-<td>64</td>
-<td>17/7/2012</td>
-</tr>
-<tr>
-<td>Papers mentioning iGEM and Registry</td>
-<td><code>("iGEM" OR "International Genetically Engineered Machine") AND("Registry of Standard Biological Parts" OR "partsregistry.org" OR "parts.mit.edu")</code></td>
-<td>330</td>
-<td>2208</td>
-<td>23</td>
-<td>42</td>
-<td>17/7/2012</td>
-</tr>
-<tr>
 <td>Papers mentioning iGEM in context of synbio</td>
 <td><code>(“synthetic biology” OR "genetic engineering") AND (“iGEM” OR “International Genetically Engineered Machine")</code></td>
@@ Line 143: / Line 86: @@
 <tr>
+<td><a href="https://docs.google.com/folder/d/0B6xW2A1PIytESjJ4SUlteG9rUEU/edit?pli=1&docId=0AqxW2A1PIytEdDVBSnY4ekt0ZUdJRXZqaFREVmJrUkE">2</a></td>
 <td>All synthetic biology</td>
 <td><code>synthetic biology</code></td>
@@ Line 153: / Line 97: @@
 <tr>
+<td><a href="https://docs.google.com/folder/d/0B6xW2A1PIytESjJ4SUlteG9rUEU/edit?pli=1&docId=0AqxW2A1PIytEdFFSV2hEMURWb1pCUmpHY3EyWndHeEE">3</a></td>
 <td>Papers mentioning Registry of Parts</td>
 <td><code>"Registry of Standard Biological Parts" OR "partsregistry.org" OR "parts.mit.edu"</code></td>
@@ Line 163: / Line 108: @@
 <tr>
+<td><a href="https://docs.google.com/folder/d/0B6xW2A1PIytESjJ4SUlteG9rUEU/edit?pli=1&docId=0AqxW2A1PIytEdFBWY3B5X0tSQl9NTUFCTDhDVGU1dWc">4</a></td>
 <td>Papers citing a particular Part</td>
 <td><code>partsregistry.org/Part:</code></td>
@@ Line 169: / Line 115: @@
 <td>5</td>
 <td>16</td>
+<td>17/7/2012</td>
+</tr>
+<tr>
+<td><a href="https://docs.google.com/folder/d/0B6xW2A1PIytESjJ4SUlteG9rUEU/edit?pli=1&docId=0AqxW2A1PIytEdFY5ay1oX0lJNEl5Ql84Y0diaUpaLXc">5</a></td>
+<td>Papers mentioning iGEM</td>
+<td><code>iGEM OR "International Genetically Engineered Machine"</code></td>
+<td>1000</td>
+<td>9095</td>
+<td>36</td>
+<td>64</td>
+<td>17/7/2012</td>
+</tr>
+<tr>
+<td><a href="https://docs.google.com/folder/d/0B6xW2A1PIytESjJ4SUlteG9rUEU/edit?pli=1&docId=0AqxW2A1PIytEdGVLRGtzLVF4NXFjOFlOR3dsRENuekE">6</a></td>
+<td>Papers mentioning iGEM and Registry</td>
+<td><code>("iGEM" OR "International Genetically Engineered Machine") AND("Registry of Standard Biological Parts" OR "partsregistry.org" OR "parts.mit.edu")</code></td>
+<td>330</td>
+<td>2208</td>
+<td>23</td>
+<td>42</td>
 <td>17/7/2012</td>
 </tr>
@@ Line 174: / Line 142: @@
 </table>
-<p>Note that searches were capped at a maximum of 1000 results. Hence getting 100 results for a query implies that <em>more exist</em>! Those first 1000 are only the ones the search engine judged most relevant.
+<p><span class="label label-info">Note</span> Searches were capped at a maximum of 1000 results. Hence getting 1000 results for a query implies that <em>more exist</em>! Those first 1000 are only the ones the search engine judged most relevant.
 </p>
+</div>
+<div class="span8 extra-spacing">
+	<h2>Why we used Google Scholar</h2>
+	<p>All in all, we found Google Scholar to most closely meet our analytic needs.</p>
+        <p>As WoK, Scopus and PubMed are strictly curated databases and limited in scope, they missed many obviously relevant publications. We also found their search options unsuitable: Many of them did not support either full text search (they looked at titles, keywords and abstracts only) or boolean operators. But for the following reasons, we needed a search to include both:
+<ul>
+<li>iGEM cannot be expected to always be the main subject of a paper, hence full text search.</li>
+<li>There are many relevant terms floating about iGEM, hence boolean operators like "OR" to allow treating papers that contain "International Genetically Engineered Machine" or the acronym "iGEM" equally.</li>
+</ul>
+        <p></p>
+	<p>Just how much wider is Google's search scope? Here is an example: PubMed gave so few results (<strong>16</strong> for <code>iGEM genetic*</code>) that we quickly discarded it. Manually merging the Web of Knowledge and Scopus results for the query <code>iGEM AND genetic*</code> (discarding obviously irrelevant results) gave <strong>43</strong> results. Then we queried Google Scholar. It gave us <strong>770</strong> for <code>(“synthetic biology” OR "genetic engineering") AND (“iGEM” OR “International Genetically Engineered Machine")</code>. </p>
+	<p>Of course, Google Scholar too is but a bronze bullet: It brings its own drawbacks. It is engineered to pick up things that only <em>seem like</em> scholarly articles. Like Google's search results in general, the results are not curated by a human. This has been criticised in the literature (<a href="http://www.emeraldinsight.com/journals.htm?articleid=1550726&show=abstract">Péter J., 2006.</a>). We found the occasional <a href="https://2012.igem.org/File:TokyoTechNotReallyNature.png">hilarious total miss</a>. Google Scholar is also known to somewhat overestimate citation counts (<a href="http://oneentry.wordpress.com/2006/08/11/research-on-citation-search-in-web-of-science-scopus-and-google-scholar/">Iselid, L., 2006.</a>). However, we concluded from empirical manual examination of a random sample that the majority of the results are plausible and (most importantly) <em>far</em> greater in scope than searches in curated databases. Taking these aspects into consideration, we found Google Scholar best fulfilled our requirements.</p>
+<p><span class="label label-warning">Caveat!</span> We only wanted to identify trends. Large and coarse pieces of data with some error were sufficient for this. These values should not be taken as exact!</p>
+</div>
+<div class="span4">
+<h2>Browse the data</h2>
+<p><form action="https://docs.google.com/folder/d/0B6xW2A1PIytESjJ4SUlteG9rUEU/edit">
+<button type="submit" class="btn btn-large btn-info" href="https://docs.google.com/folder/d/0B6xW2A1PIytESjJ4SUlteG9rUEU/edit"><img src="https://static.igem.org/mediawiki/2012/6/61/Glyphicons_090_table.png"></img> Our tables</button></form></p>
+<p>...are online in a nifty and very usable Google Docs folder.</p>
+<p>An introduction is included in case you get lost or want more information.</p>
+<h2 class="extra-spacing">On extraction tools</h2>
+<p>We made extensive use of Harzing Publish or Perish (<a href="http://www.harzing.com/pop.htm">Harzing, A.W., 2007.</a>) to scrape Google Scholar results. The tool has many limitations. However, in our experience it is the best available resource for managing the many locations in which scientific publications are deposited.</p>
+<p>We did try other things: Manual methods were too slow for hundreds of papers. Various Firefox browser plugins failed due to the search APIs having changed, were extremely awkward to use or produced clearly erroneous results. The Mac OS program <em>Papers</em> was easy to use and found large numbers of papers (as like Google Scholar it could access many sources), but had unacceptably high rates of error, problems with duplicates and could not export the results into a form we could easily process. Hence <em>Publish or Perish</em>.</p>
+</div>
+</div>
 </section> <!-- End "data-collection" -->
+<!-- Metrics
+================================================== -->
+<section id="metrics">
+<div class="page-header">
+	<h1>Metrics <small>Quantifying scientific impact</small></h1>
+</div>
+<p>Once we had scraped our data from Google Scholar, we needed a method to quantify the relevance of a given scientific article. There are many ways you can quantify success of a paper. Here are a few we investigated:</p>
+<div class="row-fluid">
+<div class="span6">
+<div class="well">
+<h3>Plain citation count</h3>
+<p>High citation count can generally be taken as an indicator of a high-quality or high-impact paper. This is the most traditional method of ranking the influence of papers.</p>
+<p>The main disadvantage of the "citation count"-method is its lack of consistent standards. Even papers within a distinct scientific fields will have differing reference approaches and citation counts. It is also significant that old papers have an edge over newer papers, as they've had more time to be cited. </p>
+</div>
+<div class="well">
+<h3>h-index</h3>
+<p>The h-index is an integer unique to a set of papers. It is used to measure the output and influence of a set of scientists. A greater h-index implies more productive and more influential authors. It was invented by physicist <a href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1283832/?tool=pmcentrez">J.E. Hirsch (2005)</a> and has since been automatically calculated by many citation databases. Here is its definition: "A set of papers has h-index <em>h</em> if <em>h</em> papers out of that set have been cited at least <em>h</em> times." An image (<a href="http://en.wikipedia.org/wiki/H-index">"Ael 2" and "Vulpecula", 2012.</a>) clarifies:
+<br /><img src="https://static.igem.org/mediawiki/2012/c/c1/HIndex.png"/></p>
+</div>
+</div>
+<div class="span6">
+<div class="well">
+<h3>g-index</h3>
+<p>The g-index is a citation index meant to quantify the influence of papers. It was proposed by <a href="http://www.springerlink.com/content/4119257t25h0852w/?MUD=MP">Leo Egghe (2006)</a> as a variation to the h-index. It puts more emphasis on the most cited papers and Egghe argues that it ranks highly cited authors more fairly. He gives the following definition: "A set of papers has a g-index <em>g</em> if <em>g</em> is the highest rank such that the top <em>g</em> papers have, together, at least <em>g²</em> citations." Here's a clarifying image by our Polish friend <a href="http://en.wikipedia.org/wiki/File:Gindex1.jpg">("Ael 2", 2012.)</a> again:
+<br /><img src="https://static.igem.org/mediawiki/2012/d/df/GIndex.png" /></p>
+</div>
+<div class="well">
+<h3>Algorithmic methods</h3>
+<p>It's worth noting that there are many other ways of quantifying productivity and impact of a set of papers or scientists. For example, <a href="http://iopscience.iop.org/1367-2630/14/3/033033/">Y.B. Zhou et al (2012)</a> propose a more complete method for "distinguishing prestige from popularity". In their algorithm, the weight of a citation to the influence of a paper is also dependent on the (already calculated) influences of the citing papers and their authors. This requires running a recursive algorithm on sufficiently complete bipartite network of papers and their authors.</p>
+</div>
+</div><!--close well-->
+</div><!--close row-->
+<div class="row-fluid">
+<div class="span12">
+<p>Though the alternatives look enticing, we ended up looking mainly at citation count.</p>
+<p>The algorithmic methods were beyond our reach of data availability: We would have had to find the names of every involved person in every iGEM team and all papers they've written, filtering out large amounts of false matches. This was impracticable.</p>
+<p>The h and g-indexes don't actually show more than the raw citation counts when it comes to tendencies over time. Also, we had relatively few search queries to compare against each other, given that two were discarded for having various flaws (discussed just below). This meant that the h and g-index, while valuable methods, were not suitable for our particular data analysis.</p>
+</div>
+</section>
 <!-- Data analysis
 ================================================== -->
 <section id="data-analysis">
-	<br>
+<div class="page-header">
-	<h1>Data analysis</h1>
+	<h1>Data analysis <small>What does this data actually mean?</small></h1>
-<h2>Some background</h2>
-<p>Here are the metrics we found for quantifying scientific influence:</p>
-<div class="accordion" id="accordion1">
-    <div class="accordion-group">
-        <div class="accordion-heading">
-            <a class="accordion-toggle" data-toggle="collapse" data-parent="#accordion1" href="#collapseOne">
-                Citations
-            </a>
-        </div>
-        <div id="collapseOne" class="accordion-body collapse" style="height: 0px; ">
-            <div class="accordion-inner">
-It's good scientific practice to mention relevant publications in one's own paper. High citation count is hence generally taken as an indicator of a high-quality or high-impact paper.
-            </div>
-        </div>
-        <div class="accordion-heading">
-            <a class="accordion-toggle" data-toggle="collapse" data-parent="#accordion1" href="#collapseTwo">
-                h-index
-            </a>
-        </div>
-        <div id="collapseTwo" class="accordion-body collapse" style="height: 0px; ">
-            <div class="accordion-inner">
-The h-index is an integer unique to a set of papers. It is used to measure the output and influence of a set of scientists. A greater h-index implies more productive and more influential authors. It was <a href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1283832/?tool=pmcentrez">invented by physicist J.E. Hirsch in 2005</a> and has since been automatically calculated by many citation databases. Here is its definition: A set of papers has h-index <em>h</em> if <em>h</em> papers out of that set have been cited at least <em>h</em> times. An image <a href="http://en.wikipedia.org/wiki/H-index">from Wikipedia</a> clarifies:<br/><img src="http://i.imgur.com/lWeHr.png"/>
-            </div>
-        </div>
-        <div class="accordion-heading">
-            <a class="accordion-toggle" data-toggle="collapse" data-parent="#accordion1" href="#collapseThree">
-                g-index
-            </a>
-        </div>
-        <div id="collapseThree" class="accordion-body collapse" style="height: 0px; ">
-            <div class="accordion-inner">
-The g-index is another citation index meant to quantify the influence of papers. It was <a href="http://www.springerlink.com/content/4119257t25h0852w/?MUD=MP">proposed in 2006 by Leo Egghe</a> as a variation to the h-index. It puts more emphasis on the most cited papers and Egghe argues that it ranks highly cited authors more fairly. He gives the following definition: "A set of papers has a g-index <em>g</em> if <em>g</em> is the highest rank such that the top <em>g</em> papers have, together, at least <em>g²</em> citations." Here's a <a href="http://en.wikipedia.org/wiki/File:Gindex1.jpg">Wikipedia-sourced diagrammatic explanation</a> again (Thanks, user "Ael 2"!):<br/><img src="http://i.imgur.com/S0NWo.png"/>
-            </div>
-        </div>
-        <div class="accordion-heading">
-            <a class="accordion-toggle" data-toggle="collapse" data-parent="#accordion1" href="#collapseFour">
-                Other methods
-            </a>
-        </div>
-        <div id="collapseFour" class="accordion-body collapse" style="height: 0px; ">
-            <div class="accordion-inner">
-<p>It's worth noting that there are many other ways of quantifying productivity and impact of a set of papers or scientists. For example, <a href="http://iopscience.iop.org/1367-2630/14/3/033033/">Y.B. Zhou et al propose a more complete method for "distinguishing prestige from popularity"</a>. In their algorithm, the weight of a citation to the influence of a paper is also dependent on the (already calculated) influences of the citing papers and their authors. This requires running a recursive algorithm on sufficiently complete bipartite network of papers and their authors. Due to unavailability of anywhere near a complete set of data (synthetic biology is a huge field) and due to the relative complexity of the algorithm, we opted for the simpler methods instead.</p>
-            </div>
-        </div>
-    </div>
 </div>
+<h2>Is our data usable?</h2>
+<h3>Yes. Mostly...</h3>
+<p>Given our doubts over accuracy of Google Scholar's data, we considered it a priority to exercise caution with our search query results. This paid off: data we compiled using the search queries of IDs 5 and 2 had fatal flaws. They were rejected from further analysis (discussed below). The other data sets were found to be suitable.</p>
+<p>Our method to examine data suitability was empirical: Random samplings of each data set were passed under human eyes. For <em>most</em> queries, this observation of random subsets showed acceptably low levels of "background static" (i.e. results that Google Scholar had automatically matched to the query, which were not actually relevant). These would form only a drop of error in the ocean of relevant data.</p>
-<h2>See the data</h2>
+<h3>...but <em>only</em> mostly.</h3>
+<p><strong>Query 5</strong> (<code>iGEM OR "International Genetically Engineered Machine"</code>) was found to have an unacceptably large level of static. The reason was quickly identified: Because the two quoted terms in Query ID 5 (<code>"iGEM"</code>, <code>"International Genetically Engineered Machine"</code>) were separated by a disjunction (<code>OR</code>), the query would easily match anything that contained just the acronym "IGEM"! This meant acronyms in economics such as "Inter-temporal General Equilibrium Model (IGEM)" or the British "Institution of Gas Engineers & Managers (IGEM)" and various medical terms and chemical names snuck in. The entire data set with ID 5 was dismissed from further analysis.</p>
+<p>The lesson we took from this is not to search for short acronyms by themselves. Query 1 (<code>(“synthetic biology” OR "genetic engineering") AND (“iGEM” OR “International Genetically Engineered Machine")</code>) could be thought of as the "Version Two" of the problematic Query 5. It searches for the same terms, but includes a conjunction (<code>AND</code>) with either <em>synthetic biology</em> or <em>genetic engineering</em>, which bends results toward <em>our</em> iGEM. This tunes static down to an acceptable level.</p>
-<div class="well span4" style="float:none"><!-- float:none so other text doesn't even try to wrap around this -->
+<p><strong>Query 2</strong> (<code>synthetic biology</code>) had a different problem: It was too big. The query was an attempt to capture stats for the entire field of synthetic biology, so we could statistically determine the relative influence of the iGEM competition. However, we had forgotten the 1000-result cap imposed by Google Scholar. It is impossible to retrieve results beyond this 1k "event horizon". Google does not publish information regarding how the order of results is determined. Hence these first 1000 results (out of what are likely to be 10s or 100s of thousands of papers) are all biased by some unknown force. Were more cited papers favoured? Were papers published more recently favoured? No conclusions can be drawn from a biased and small subset of the full data. We also discounted data set 2 from any further analysis.</p>
-<p><a class="btn btn-large" href="https://docs.google.com/folder/d/0B6xW2A1PIytESjJ4SUlteG9rUEU/edit">All Our Data</a></p>
-<p>...is online in a nifty Google Docs folder (as you'll see if you click there). Each set of data is a table with the results of a query. An introduction is included in case you get lost or want more information.</p></div>
-<p><span style="background:#ffff00"> TODO: Continue here :) </span></p>
-</section>
-<section id="old-data">
+<p>To emphasize: Data sets 2 and 5 are <em>not included</em> in any further analysis.</p>
-<h1>Old data (in case it still becomes useful)</h1>
-<div class="accordion" id="accordion2">
-            <div class="accordion-group">
-              <div class="accordion-heading">
-                <a class="accordion-toggle" data-toggle="collapse" data-parent="#accordion2" href="#collapseOne">
-                  Elsevier's Scopus database, search query "iGEM". Irrelevant results (e.g. Institution of Gas Engineers and Managers) manually filtered. Data collection performed on 26 June 2012.
-                </a>
-              </div>
-              <div id="collapseOne" class="accordion-body collapse" style="height: 0px; ">
-                <div class="accordion-inner">
-<table class="table table-striped">
-	<tr>
-		<td >Authors</td>
-		<td >Title</td>
-		<td >Year</td>
-		<td >Source title</td>
-		<td >Cited by</td>
-	</tr>
-	<tr>
-		<td >Boyle P.M.; Burrill D.R.; Inniss M.C.; Agapakis C.M.; Deardon A.; DeWerd J.G.; Gedeon M.A.; Quinn J.Y.; Paull M.L.; Raman A.M.; Theilmann M.R.; Wang L.; Winn J.C.; Medvedik O.; Schellenberg K.; Haynes K.A.; Viel A.; Brenner T.J.; Church G.M.; Shah J.V.; Silver P.A.</td>
-		<td >A BioBrick compatible strategy for genetic modification of plants</td>
-		<td >2012</td>
-		<td >Journal of Biological Engineering</td>
-		<td ></td>
-	</tr>
-	<tr>
-		<td >Hesselman M.C.; Odoni D.I.; Ryback B.M.; de Groot S.; van Heck R.G.A.; Keijsers J.; Kolkman P.; Nieuwenhuijse D.; van Nuland Y.M.; Sebus E.; Spee R.; de Vries H.; Wapenaar M.T.; Ingham C.J.; Schroen K.; Martins dos Santos V.A.P.; Spaans S.K.; Hugenholtz F.; van Passel M.W.J.</td>
-		<td >A multi-platform flow device for microbial (co-) cultivation and microscopic analysis</td>
-		<td >2012</td>
-		<td >PLoS ONE</td>
-		<td ></td>
-	</tr>
-	<tr>
-		<td >Materi W.</td>
-		<td >Leading a successful iGEM team</td>
-		<td >2012</td>
-		<td >Methods in Molecular Biology</td>
-		<td ></td>
-	</tr>
-	<tr>
-		<td >Ho-Shing O.; Lau K.H.; Vernon W.; Eckdahl T.T.; Campbell A.M.</td>
-		<td >Assembly of standardized DNA parts using biobrick ends in E. coli</td>
-		<td >2012</td>
-		<td >Methods in Molecular Biology</td>
-		<td ></td>
-	</tr>
-	<tr>
-		<td >Giavitto J.-L.</td>
-		<td >The modeling and the simulation of the fluid machines of synthetic biology</td>
-		<td >2012</td>
-		<td >Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)</td>
-		<td ></td>
-	</tr>
-	<tr>
-		<td >Muller K.M.; Arndt K.M.</td>
-		<td >Standardization in synthetic biology</td>
-		<td >2012</td>
-		<td >Methods in Molecular Biology</td>
-		<td ></td>
-	</tr>
-	<tr>
-		<td >Kuldell N.</td>
-		<td >Living machines: Some assembly required: Kit-based competitions challenge teams of students to learn microbiology and design principles in the context of synthetic biology</td>
-		<td >2012</td>
-		<td >Microbe</td>
-		<td ></td>
-	</tr>
-	<tr>
-		<td >Stewart D.; Wilson-Kanamori J.R.</td>
-		<td >Modular modelling in synthetic biology: Light-based communication in E. coli</td>
-		<td >2011</td>
-		<td >Electronic Notes in Theoretical Computer Science</td>
-		<td ></td>
-	</tr>
-	<tr>
-		<td >Dixon J.; Kuldell N.</td>
-		<td >BioBuilding: Using banana-scented bacteria to teach synthetic biology</td>
-		<td >2011</td>
-		<td >Methods in Enzymology</td>
-		<td >1</td>
-	</tr>
-	<tr>
-		<td >Mitchell R.; Dori Y.J.; Kuldell N.H.</td>
-		<td >Experiential Engineering Through iGEM-An Undergraduate Summer Competition in Synthetic Biology</td>
-		<td >2011</td>
-		<td >Journal of Science Education and Technology</td>
-		<td ></td>
-	</tr>
-	<tr>
-		<td >Dress L.</td>
-		<td >iGEM 2010: Synthetic biologists compete for the future</td>
-		<td >2010</td>
-		<td >Industrial Biotechnology</td>
-		<td ></td>
-	</tr>
-	<tr>
-		<td >Hafner S.</td>
-		<td >IGEM 2009: Synthethic biology and ethics [Les iGEM 2009 la biologie synth-Ã©thique]</td>
-		<td >2010</td>
-		<td >Medecine/Sciences</td>
-		<td >2</td>
-	</tr>
-	<tr>
-		<td >Gu X.; Trybilo M.; Ramsay S.; Jensen M.; Fulton R.; Rosser S.; Gilbert D.</td>
-		<td >Engineering a novel self-powering electrochemical biosensor</td>
-		<td >2010</td>
-		<td >Systems and Synthetic Biology</td>
-		<td ></td>
-	</tr>
-	<tr>
-		<td >Cai Y.; Wilson M.L.; Peccoud J.</td>
-		<td >GenoCAD for iGEM: A grammatical approach to the design of standard-compliant constructs</td>
-		<td >2010</td>
-		<td >Nucleic Acids Research</td>
-		<td >12</td>
-	</tr>
-	<tr>
-		<td >Cooling M.T.; Rouilly V.; Misirli G.; Lawson J.; Yu T.; Hallinan J.; Wipat A.</td>
-		<td >Standard virtual biological parts: A repository of modular modeling components for synthetic biology</td>
-		<td >2010</td>
-		<td >Bioinformatics</td>
-		<td >10</td>
-	</tr>
-	<tr>
-		<td >Smolke C.D.</td>
-		<td >Building outside of the box: IGEM and the BioBricks Foundation</td>
-		<td >2009</td>
-		<td >Nature Biotechnology</td>
-		<td >15</td>
-	</tr>
-	<tr>
-		<td >Hafner S.</td>
-		<td >iGEM competition 2008; two French teams! [CompÃ©tition iGEM 2008: Deux Ã©quipes franÃ§aises!]</td>
-		<td >2009</td>
-		<td >Medecine/Sciences</td>
-		<td >2</td>
-	</tr>
-	<tr>
-		<td >Jerala R.</td>
-		<td >I like iGEM</td>
-		<td >2009</td>
-		<td >Scientist</td>
-		<td ></td>
-	</tr>
-	<tr>
-		<td >Edwards C.</td>
-		<td >The gene machines</td>
-		<td >2008</td>
-		<td >Engineering and Technology</td>
-		<td ></td>
-	</tr>
-	<tr>
-		<td >Bikard D.; Kepes F.</td>
-		<td >First French team success during iGEM Synthetic biology competition [SuccÃ¨s de la premiÃ¨re Ã©quipe franÃ§aise lors de la compÃ©tition iGEM de biologie synthÃ©tique]</td>
-		<td >2008</td>
-		<td >Medecine/Sciences</td>
-		<td >6</td>
-	</tr>
-	<tr>
-		<td >Bachman R.</td>
-		<td >An iGEM of an idea? [2]</td>
-		<td >2008</td>
-		<td >Scientist</td>
-		<td ></td>
-	</tr>
-	<tr>
-		<td >Kaczkowski P.</td>
-		<td >An iGEM of an idea? [1]</td>
-		<td >2008</td>
-		<td >Scientist</td>
-		<td ></td>
-	</tr>
-	<tr>
-		<td >Goodman C.</td>
-		<td >Engineering ingenuity at iGEM</td>
-		<td >2008</td>
-		<td >Nature Chemical Biology</td>
-		<td >16</td>
-	</tr>
-	<tr>
-		<td >Gallagher R.</td>
-		<td >An iGEM of an idea</td>
-		<td >2007</td>
-		<td >Scientist</td>
-		<td >2</td>
-	</tr>
-	<tr>
-		<td >Brown J.</td>
-		<td >The iGEM competition: Building with biology</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >7</td>
-	</tr>
-	<tr>
-		<td >Haynes K.A.; Broderick M.L.; Brown A.D.; Butner T.L.; Harden L.; Heard L.; Jessen E.; Malloy K.; Ogden B.; Rosemond S.; Simpson S.; Zwack E.; Campbell A.M.; Eckdahl T.; Heyer L.J.; Poet J.L.</td>
-		<td >Computing with living hardware</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >2</td>
-	</tr>
-	<tr>
-		<td >Dabholkar S.; Thattai M.</td>
-		<td >Brainstorming biology</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >1</td>
-	</tr>
-	<tr>
-		<td >Brown T.; Chang C.; Heinze B.; Hollinger P.; Kittleson J.; MacDow K.; Reavis D.; Curry J.; Riley M.</td>
-		<td >Development of an inducible three colour bacterial water colour system</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >1</td>
-	</tr>
-	<tr>
-		<td >Rodrigo G.; Montagud A.; Aparici A.; Aroca M.C.; Baguena M.; Carrera J.; Edo C.; Fernandez-De-Cordoba P.; Ferrando A.; Fuertes G.; Gimenez D.; Mata C.; Medrano J.V.; Navarrete C.; Navarro E.; Salgado J.; Tortosa P.; Urchueguia J.; Jaramillo A.</td>
-		<td >Vanillin cell sensor</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >1</td>
-	</tr>
-	<tr>
-		<td >Quiroz J.N.A.; Flores R.B.; Cisneros T.G.B.; Rosales I.Y.F.; Naranjo A.G.; Sanchez J.C.G.; Jimenez M.E.G.; Padilla R.E.G.; Baena A.J.L.; Hernandez P.A.L.; Padilla P.G.; Miller R.P.; Gaspar I.N.R.; Chico J.C.R.; Martinez A.R.; Romero J.P.; Arzate A.S.; Barradas J.S.A.; Diaz D.A.; Bracho A.B.; Benitez C.; Arteag C.I.F.; Quiroz F.H.; Martinez G.J.; Rabadan J.L.; Salvador M.C.O.; Longoria P.P.; Orozco R.P.; Corona F.R.; Majarrez E.S.; Hassan E.S.; Sanchez C.S.; Saldana U.V.; Segura P.B.Z.</td>
-		<td >Biological implementation of algorithms and unconventional computing</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td ></td>
-	</tr>
-	<tr>
-		<td >King P.; Lavrovsky V.; Von Mammen S.; Jacob C.</td>
-		<td >Teaching bacteria how to dance</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td ></td>
-	</tr>
-	<tr>
-		<td >Badalamenti J.P.; Weiss L.E.; Buckno C.J.; Richard T.L.; Weiss P.S.; Cirino P.C.</td>
-		<td >Synthetic sports: A bacterial relay race</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >1</td>
-	</tr>
-	<tr>
-		<td >Lohmueller J.; Neretti N.; Hickey B.; Kaka A.; Gao A.; Lemon J.; Lattanzi V.; Goldstein P.; Tam L.-K.; Schmidt M.; Brodsky A.S.; Haberstroh K.; Morgan J.; Palmore T.; Wessel G.; Jaklenec A.; Urabe H.; Gagnon J.; Cumbers J.</td>
-		<td >Progress toward construction and modelling of a tri-stable toggle switch in E. coli</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >2</td>
-	</tr>
-	<tr>
-		<td >Ravi M.; Ngeleka M.; Kim S.-H.; Gyles C.; Berthiaume F.; Mourez M.; Middleton D.; Simko E.</td>
-		<td >Contribution of AIDA-I to the pathogenicity of a porcine diarrheagenic Escherichia coli and to intestinal colonization through biofilm formation in pigs</td>
-		<td >2007</td>
-		<td >Veterinary Microbiology</td>
-		<td >10</td>
-	</tr>
-</table>                </div>
-              </div>
-            </div>
-            <div class="accordion-group">
-              <div class="accordion-heading">
-                <a class="accordion-toggle" data-toggle="collapse" data-parent="#accordion2" href="#collapseTwo">
-                  Reuters' Web of Knowledge database, search query "iGEM". Irrelevant results (e.g. Institution of Gas Engineers and Managers) manually filtered. Data collection performed on 02 July 2012. Citation data not extracted.
-                </a>
-              </div>
-              <div id="collapseTwo" class="accordion-body in collapse" style="height: 0px; ">
-                <div class="accordion-inner">
-                	<table class="table table-striped">
+<h2>What's is iGEM's impact?</h2>
-	<tr>
+<p>What does "impact" mean within a scientific context? We will operate under the assumption that <em>having impact</em> correlates strongly with <em>being mentioned</em> in scholarly articles. Hence, we quantify the impact that a term is having simply by searching for that term, and summing up result counts.</p>
-		<td >Authors</td>
+<p>Here is a chart summarising how many papers mention various terms floating about iGEM over time:</p>
-		<td >Title</td>
+<div class="row-fluid">
-		<td >Year</td>
+<div class="span8">
-		<td >Source title</td>
+    <a class="thumbnail" href="#modal-attention-summary" data-toggle="modal">
-		<td >Cited by</td>
+      <img src="https://static.igem.org/mediawiki/2012/a/af/DataSummaryStats.png" alt="">
-	</tr>
+<p class="label-set">
-	<tr>
+<span class="label" style="background-color:#ffaf2b">Number of teams</span>
-		<td >Cooling, M. T.; Rouilly, V.; Misirli, G.; Lawson, J.; Yu, T.; Hallinan, J.; Wipat, A.</td>
+<span class="label" style="background-color:#d2eaa2;color:#353535">Parts submitted (10s)</span>
-		<td >Standard virtual biological parts: a repository of modular modeling components for synthetic biology</td>
+<span class="label" style="background-color:#81ccf1">Papers mentioning iGEM</span>
-		<td >2010</td>
+<span class="label" style="background-color:#7fb02b">Papers mentioning Registry of Parts</span>
-		<td >BIOINFORMATICS</td>
+<span class="label" style="background-color:#cc6200">Papers mentioning specific Registry Parts</span></p>
-		<td >Not collected</td>
+      <h5>Chart 1: <em>Summary of iGEM-related attention over time</em></h5>
-	</tr>
+      <p>This chart shows the amount of "attention" the iGEM competition is receiving over time, shown relative to the growth of the competition itself (quantified by amount of registered teams). "Attention" is measured by amount of papers mentioning certain keywords deemed to be relevant to iGEM or the Registry of Standard Parts. An overall positive trend in proportion with the growth of the competition can be seen.</p>
-	<tr>
+</a>
-		<td >Mitchell, Rudolph; Dori, Yehudit Judy; Kuldell, Natalie H.</td>
+</div>
-		<td >Experiential Engineering Through iGEM-An Undergraduate Summer Competition in Synthetic Biology</td>
-		<td >2011</td>
-		<td >JOURNAL OF SCIENCE EDUCATION AND TECHNOLOGY</td>
-		<td >Not collected</td>
-	</tr>
-	<tr>
-		<td >Haefner, Sophia</td>
-		<td >iGEM competition 2008, two French teams!</td>
-		<td >2009</td>
-		<td >M S-MEDECINE SCIENCES</td>
-		<td >Not collected</td>
-	</tr>
-	<tr>
-		<td >Bikard, David; Kepes, Francois</td>
-		<td >First French team success during iGEM synthetic biology competition</td>
-		<td >2008</td>
-		<td >M S-MEDECINE SCIENCES</td>
-		<td >Not collected</td>
-	</tr>
-	<tr>
-		<td >Hafner, Sophia</td>
-		<td >[IGEM 2009: synthetic biology and ethics].</td>
-		<td >2010</td>
-		<td >Medecine sciences : M/S</td>
-		<td >Not collected</td>
-	</tr>
-	<tr>
-		<td >Dixon, James; Kuldell, Natalie</td>
-		<td >BIOBUILDING: USING BANANA-SCENTED BACTERIA TO TEACH SYNTHETIC BIOLOGY</td>
-		<td >2011</td>
-		<td >METHODS IN ENZYMOLOGY, VOL 497: SYNTHETIC BIOLOGY, METHODS FOR PART/DEVICE CHARACTERIZATION AND CHASSIS ENGINEERING, PT A</td>
-		<td >Not collected</td>
-	</tr>
-	<tr>
-		<td >Ho-Shing, Olivia; Lau, Kin H; Vernon, William; Eckdahl, Todd T; Campbell, A Malcolm</td>
-		<td >Assembly of standardized DNA parts using BioBrick ends in E. coli.</td>
-		<td >2012</td>
-		<td >Methods in molecular biology (Clifton, N.J.)</td>
-		<td >Not collected</td>
-	</tr>
-	<tr>
-		<td >Materi, Wayne</td>
-		<td >Leading a successful iGEM team.</td>
-		<td >2012</td>
-		<td >Methods in molecular biology (Clifton, N.J.)</td>
-		<td >Not collected</td>
-	</tr>
-	<tr>
-		<td >Muller, Kristian M; Arndt, Katja M</td>
-		<td >Standardization in synthetic biology.</td>
-		<td >2012</td>
-		<td >Methods in molecular biology (Clifton, N.J.)</td>
-		<td >Not collected</td>
-	</tr>
-	<tr>
-		<td >Smolke, Christina D.</td>
-		<td >Building outside of the box: iGEM and the BioBricks Foundation</td>
-		<td >2009</td>
-		<td >NATURE BIOTECHNOLOGY</td>
-		<td >Not collected</td>
-	</tr>
-	<tr>
-		<td >Goodman, Catherine</td>
-		<td >Engineering ingenuity at iGEM</td>
-		<td >2008</td>
-		<td >NATURE CHEMICAL BIOLOGY</td>
-		<td >Not collected</td>
-	</tr>
-	<tr>
-		<td >Cai, Yizhi; Wilson, Mandy L.; Peccoud, Jean</td>
-		<td >GenoCAD for iGEM: a grammatical approach to the design of standard-compliant constructs</td>
-		<td >2010</td>
-		<td >NUCLEIC ACIDS RESEARCH</td>
-		<td >Not collected</td>
-	</tr>
-	<tr>
-		<td >Hesselman, Matthijn C; Odoni, Dorett I; Ryback, Brendan M; de Groot, Suzette; van Heck, Ruben G A; Keijsers, Jaap; Kolkman, Pim; Nieuwenhuijse, David; van Nuland, Youri M; Sebus, Erik; Spee, Rob; de Vries, Hugo; Wapenaar, Marten T; Ingham, Colin J; Schroen, Karin; Martins Dos Santos, Vitor A P; Spaans, Sebastiaan K; Hugenholtz, Floor; van Passel, Mark W J</td>
-		<td >A multi-platform flow device for microbial (co-) cultivation and microscopic analysis.</td>
-		<td >2012</td>
-		<td >PloS one</td>
-		<td >Not collected</td>
-	</tr>
-	<tr>
-		<td >Vinson, Valda</td>
-		<td >iGEM 2010 Jamboree</td>
-		<td >2010</td>
-		<td >SCIENCE</td>
-		<td >Not collected</td>
-	</tr>
-	<tr>
-		<td >Jerala, Roman</td>
-		<td >I like iGEM</td>
-		<td >2009</td>
-		<td >SCIENTIST</td>
-		<td >Not collected</td>
-	</tr>
-	<tr>
-		<td >Bachman, Robert</td>
-		<td >An iGEM of an idea?</td>
-		<td >2008</td>
-		<td >SCIENTIST</td>
-		<td >Not collected</td>
-	</tr>
-	<tr>
-		<td >Kaczkowski, Peter</td>
-		<td >An iGEM of an idea?</td>
-		<td >2008</td>
-		<td >SCIENTIST</td>
-		<td >Not collected</td>
-	</tr>
-	<tr>
-		<td >Gallagher, Richard</td>
-		<td >An iGEM of an Idea - How to get students to go into science</td>
-		<td >2007</td>
-		<td >SCIENTIST</td>
-		<td >Not collected</td>
-	</tr>
-	<tr>
-		<td >Mueller, Kristian M.; Arndt, Katja M.</td>
-		<td >Standardization in Synthetic Biology</td>
-		<td >2011</td>
-		<td >SYNTHETIC GENE NETWORKS: METHODS AND PROTOCOLS</td>
-		<td >Not collected</td>
-	</tr>
-	<tr>
-		<td >Gu, X; Trybilo, M; Ramsay, S; Jensen, M; Fulton, R; Rosser, S; Gilbert, D</td>
-		<td >Engineering a novel self-powering electrochemical biosensor.</td>
-		<td >2010</td>
-		<td >Systems and synthetic biology</td>
-		<td >Not collected</td>
-	</tr>
-</table>
-                </div>
+<div class="span4">
-              </div>
+<h3>And the answer is...</h3>
-            </div>
+<p>Things are looking good for iGEM! Since the first iGEM competition in 2003, more teams are participating each year and more parts are being submitted. The efforts to expose iGEM to the community has paid off - the amount of scientific papers mentioning iGEM and the Registry of Standard Parts has risen, proportional to the increase of the competition itself.</p>
-            <div class="accordion-group">
+<p>Papers mentioning a specific Registry BioBrick have only begun to appear in recent years, but the numbers show growth. We hypothesize that the Registry's contents is only now reaching the critical mass to become a useful research tool. The founders' dream (<a href="https://igem.org/Press_Kit">iGEM Foundation, 2012b</a>) of "making genetics modular" is becoming reality.</p>
-              <div class="accordion-heading">
-                <a class="accordion-toggle" data-toggle="collapse" data-parent="#accordion2" href="#collapseThree">
-                  Journal of Biological Engineering. Data collection performed on 02 July 2012. Recommended by iGEM.
-                </a>
-              </div>
-              <div id="collapseThree" class="accordion-body collapse" style="height: 0px; ">
-                <div class="accordion-inner">
-                  <table class="table table-striped">
-	<tr>
-		<td >Authors</td>
-		<td >Title</td>
-		<td >Year</td>
-		<td >Source title</td>
-		<td >Cited by</td>
-	</tr>
-	<tr>
-		<td >Anderson, J. Christopher, John Dueber, Mariana Leguia, Gabriel Wu, Jonathan Goler, Adam Arkin, and Jay Keasling.</td>
-		<td >BglBricks: A Flexible Standard for Biological Part Assembly.</td>
-		<td >2010</td>
-		<td >Journal of Biological Engineering</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >Baumgardner, Jordan, Karen Acker, Oyinade Adefuye, Samuel Crowley, Will DeLoache, James Dickson, Lane Heard, et al.</td>
-		<td >Solving a Hamiltonian Path Problem with a Bacterial Computer.</td>
-		<td >2009</td>
-		<td >Journal of Biological Engineering</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >Chen, Joanna, Douglas Densmore, Timothy Ham, Jay Keasling, and Nathan Hillson.</td>
-		<td >DeviceEditor Visual Biological CAD Canvas.</td>
-		<td >2012</td>
-		<td >Journal of Biological Engineering</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >Hatch, Alex.</td>
-		<td >Four Suggestions for Addressing Public Concern Regarding Synthetic Biology</td>
-		<td >2010</td>
-		<td >Journal of Biological Engineering</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >Haynes, Karmella, Marian Broderick, Adam Brown, Trevor Butner, James Dickson, W. Lance Harden, Lane Heard, et al.</td>
-		<td >Engineering Bacteria to Solve the Burnt Pancake Problem.</td>
-		<td >2008</td>
-		<td >Journal of Biological Engineering</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >Kelly, Jason, Adam Rubin, Joseph Davis, Caroline Ajo-Franklin, John Cumbers, Michael Czar, Kim de Mora, Aaron Glieberman, Dileep Monie, and Drew Endy.</td>
-		<td >Measuring the Activity of BioBrick Promoters Using an in Vivo Reference Standard.</td>
-		<td >2009</td>
-		<td >Journal of Biological Engineering</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >Kuldell, Natalie.</td>
-		<td >Authentic Teaching and Learning Through Synthetic Biology.</td>
-		<td >2007</td>
-		<td >Journal of Biological Engineering</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >Norville, Julie, Ratmir Derda, Saurabh Gupta, Kelly Drinkwater, Angela Belcher, Andres Leschziner, and Thomas Knight. </td>
-		<td >Introduction of Customized Inserts for Streamlined Assembly and Optimization of BioBrick Synthetic Genetic Circuits.</td>
-		<td >2010</td>
-		<td >Journal of Biological Engineering</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >Pasotti, Lorenzo, Susanna Zucca, Manuel Lupotto, Maria Cusella De Angelis, and Paolo Magni.</td>
-		<td >Characterization of a Synthetic Bacterial Self-destruction Device for Programmed Cell Death and for Recombinant Proteins Release.</td>
-		<td >2011</td>
-		<td >Journal of Biological Engineering</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >Pearson, Brianna, Sam Snell, Kyri Bye-Nagel, Scott Tonidandel, Laurie Heyer, and A. Malcolm Campbell.</td>
-		<td >Word Selection Affects Perceptions of Synthetic Biology.</td>
-		<td >2011</td>
-		<td >Journal of Biological Engineering</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >Shetty, Reshma, Drew Endy, and Thomas Knight.</td>
-		<td >Engineering BioBrick Vectors from BioBrick Parts.</td>
-		<td >2008</td>
-		<td >Journal of Biological Engineering</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >Speer, Michael, and Tom Richard.</td>
-		<td >Amplified Insert Assembly: An Optimized Approach to Standard Assembly of BioBrickTM Genetic Circuits.</td>
-		<td >2011</td>
-		<td >Journal of Biological Engineering</td>
-		<td >No data</td>
-	</tr>
-</table>
-                </div>
-              </div>
-            </div>
-            <div class="accordion-group">
-              <div class="accordion-heading">
-                <a class="accordion-toggle" data-toggle="collapse" data-parent="#accordion2" href="#collapseFour">
-                  Institution of Engineering and Technology. Data collection performed on 02 July 2012. Recommended by iGEM.
-                </a>
-              </div>
-              <div id="collapseFour" class="accordion-body collapse" style="height: 0px; ">
-                <div class="accordion-inner">
-                  <table class="table table-striped">
-	<tr>
-		<td >Authors</td>
-		<td >Title</td>
-		<td >Year</td>
-		<td >Source title</td>
-		<td >Cited by</td>
-	</tr>
-	<tr>
-		<td >J. Brown</td>
-		<td >The iGEM competition: building with biology</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >P. King</td>
-		<td >Teaching bacteria how to dance</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >J.P. Badalamenti</td>
-		<td >Synthetic sports: a bacterial relay race</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >S. Dabholkar</td>
-		<td >Brainstorming biology</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >T. Brown</td>
-		<td >Development of an inducible three colour bacterial water colour system</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >J. Lohmueller</td>
-		<td >Progress toward construction and modelling of a tri-stable toggle switch in E. coli</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >K.A. Haynes</td>
-		<td >Computing with living hardware</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >J.N. Argota Quiróz</td>
-		<td >Biological implementation of algorithms and unconventional computing</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >G. Rodrigo</td>
-		<td >Vanillin cell sensor</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >No data</td>
-	</tr>
-</table>
-                </div>
-              </div>
-            </div>
-           <div class="accordion-group">
+<h3>Where this data comes from</h3>
-              <div class="accordion-heading">
+<p>Data about participating teams and number of submitted BioBricks comes from the <a href="https://igem.org/Previous_iGEM_Competitions">iGEM Foundation (2012a)</a>. The other data sets come from the results Google Scholar queries with IDs 1, 3 and 4 (see query summary table above).</p>
-                <a class="accordion-toggle" data-toggle="collapse" data-parent="#accordion2" href="#collapseFive">
+</div>
-                  <b>Combinated dataset.</b> All duplicates removed.
+</div>
-                </a>
+<h2>Is the relationship between iGEM and the <abbr title="Registry of Standard Biological Parts">Registry</abbr> clear?</h2>
-              </div>
+<div class="row-fluid">
-              <div id="collapseFive" class="accordion-body collapse" style="height: 0px; ">
+<div class="span12 well">
-                <div class="accordion-inner">
+    <ul class="thumbnails">
-                  <table class="table table-striped">
+    <li class="span4">
-                  		<tr>
+    <a href="#modal-igem-attention" data-toggle="modal" class="thumbnail">
-		<td >Authors</td>
+    <img src="https://static.igem.org/mediawiki/2012/1/12/Data1Chart_ignoresBefore2k.png" alt="">
-		<td >Title</td>
+    <p class="label-set">
-		<td >Year</td>
+    <span class="label" style="background-color:#ff7900">Papers published in that year</span>
-		<td >Source title</td>
+    <span class="label" style="background-color:#b0db54">Times those papers were cited</span>
-		<td >Cited by</td>
+    </p>
-	</tr>
+    <p><h5>Chart 2: <em>"iGEM" attention</em></h5></p>
-	<tr>
+    </a>
-		<td >Kuldell, Natalie.</td>
+    </li>
-		<td >Authentic Teaching and Learning Through Synthetic Biology.</td>
+    <li class="span4">
-		<td >2007</td>
+    <a href="#modal-registry-attention" data-toggle="modal" class="thumbnail">
-		<td >Journal of Biological Engineering</td>
+    <img src="https://static.igem.org/mediawiki/2012/3/3f/Data3Chart.png" >
-		<td >No data</td>
+    <p class="label-set">
-	</tr>
+    <span class="label" style="background-color:#ff7900">Papers published in that year</span>
-	<tr>
+    <span class="label" style="background-color:#b0db54">Times those papers were cited</span>
-		<td >Haynes, Karmella, Marian Broderick, Adam Brown, Trevor Butner, James Dickson, W. Lance Harden, Lane Heard, et al.</td>
+    </p>
-		<td >Engineering Bacteria to Solve the Burnt Pancake Problem.</td>
+    <p><h5>Chart 3: <em>"Registry" attention</em></h5></p>
-		<td >2008</td>
+    </a>
-		<td >Journal of Biological Engineering</td>
+    </li>
-		<td >No data</td>
+    <li class="span4">
-	</tr>
+    <a href="#modal-igem-registry-attention" data-toggle="modal" class="thumbnail">
-	<tr>
+    <img src="https://static.igem.org/mediawiki/2012/e/ee/Data6Chart.png" alt="">
-		<td >Shetty, Reshma, Drew Endy, and Thomas Knight.</td>
+    <p class="label-set">
-		<td >Engineering BioBrick Vectors from BioBrick Parts.</td>
+    <span class="label" style="background-color:#ff7900">Papers published in that year</span>
-		<td >2008</td>
+    <span class="label" style="background-color:#b0db54">Times those papers were cited</span>
-		<td >Journal of Biological Engineering</td>
+    </p>
-		<td >No data</td>
+    <p><h5>Chart 4: <em>"iGEM and Registry" attention</em></h5></p>
-	</tr>
+    </a>
-	<tr>
+    </li>
-		<td >Baumgardner, Jordan, Karen Acker, Oyinade Adefuye, Samuel Crowley, Will DeLoache, James Dickson, Lane Heard, et al.</td>
+    </ul>
-		<td >Solving a Hamiltonian Path Problem with a Bacterial Computer.</td>
+<p>These three graphs show the amount of papers published each year containing certain search queries, as well as the number of times these papers were cited. All graphs show positive tendencies: the competition is becoming more wide-spread and more iGEM-related papers are being published and recognized. The search queries were chosen to show which part of iGEM is usually cited: the iGEM competition, the Registry of Standard Parts, or both. The data shows that only around half of papers will cite both elements. Some far outlying data points (clear errors) were not plotted to maintain uniform scale.</p>
-		<td >2009</td>
+</div>
-		<td >Journal of Biological Engineering</td>
+</div>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >Kelly, Jason, Adam Rubin, Joseph Davis, Caroline Ajo-Franklin, John Cumbers, Michael Czar, Kim de Mora, Aaron Glieberman, Dileep Monie, and Drew Endy.</td>
-		<td >Measuring the Activity of BioBrick Promoters Using an in Vivo Reference Standard.</td>
-		<td >2009</td>
-		<td >Journal of Biological Engineering</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >Anderson, J. Christopher, John Dueber, Mariana Leguia, Gabriel Wu, Jonathan Goler, Adam Arkin, and Jay Keasling.</td>
-		<td >BglBricks: A Flexible Standard for Biological Part Assembly.</td>
-		<td >2010</td>
-		<td >Journal of Biological Engineering</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >Hatch, Alex.</td>
-		<td >Four Suggestions for Addressing Public Concern Regarding Synthetic Biology</td>
-		<td >2010</td>
-		<td >Journal of Biological Engineering</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >Norville, Julie, Ratmir Derda, Saurabh Gupta, Kelly Drinkwater, Angela Belcher, Andres Leschziner, and Thomas Knight. </td>
-		<td >Introduction of Customized Inserts for Streamlined Assembly and Optimization of BioBrick Synthetic Genetic Circuits.</td>
-		<td >2010</td>
-		<td >Journal of Biological Engineering</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >Pasotti, Lorenzo, Susanna Zucca, Manuel Lupotto, Maria Cusella De Angelis, and Paolo Magni.</td>
-		<td >Characterization of a Synthetic Bacterial Self-destruction Device for Programmed Cell Death and for Recombinant Proteins Release.</td>
-		<td >2011</td>
-		<td >Journal of Biological Engineering</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >Pearson, Brianna, Sam Snell, Kyri Bye-Nagel, Scott Tonidandel, Laurie Heyer, and A. Malcolm Campbell.</td>
-		<td >Word Selection Affects Perceptions of Synthetic Biology.</td>
-		<td >2011</td>
-		<td >Journal of Biological Engineering</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >Speer, Michael, and Tom Richard.</td>
-		<td >Amplified Insert Assembly: An Optimized Approach to Standard Assembly of BioBrickTM Genetic Circuits.</td>
-		<td >2011</td>
-		<td >Journal of Biological Engineering</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >Chen, Joanna, Douglas Densmore, Timothy Ham, Jay Keasling, and Nathan Hillson.</td>
-		<td >DeviceEditor Visual Biological CAD Canvas.</td>
-		<td >2012</td>
-		<td >Journal of Biological Engineering</td>
-		<td >No data</td>
-	</tr>
-	<tr>
-		<td >Vinson, Valda</td>
-		<td >iGEM 2010 Jamboree</td>
-		<td >2010</td>
-		<td >SCIENCE</td>
-		<td >Not collected</td>
-	</tr>
-	<tr>
-		<td >Goodman C.</td>
-		<td >Engineering ingenuity at iGEM</td>
-		<td >2008</td>
-		<td >Nature Chemical Biology</td>
-		<td >16</td>
-	</tr>
-	<tr>
-		<td >Smolke C.D.</td>
-		<td >Building outside of the box: IGEM and the BioBricks Foundation</td>
-		<td >2009</td>
-		<td >Nature Biotechnology</td>
-		<td >15</td>
-	</tr>
-	<tr>
-		<td >Cai Y., Wilson M.L., Peccoud J.</td>
-		<td >GenoCAD for iGEM: A grammatical approach to the design of standard-compliant constructs</td>
-		<td >2010</td>
-		<td >Nucleic Acids Research</td>
-		<td >12</td>
-	</tr>
-	<tr>
-		<td >Ravi M., Ngeleka M., Kim S.-H., Gyles C., Berthiaume F., Mourez M., Middleton D., Simko E.</td>
-		<td >Contribution of AIDA-I to the pathogenicity of a porcine diarrheagenic Escherichia coli and to intestinal colonization through biofilm formation in pigs</td>
-		<td >2007</td>
-		<td >Veterinary Microbiology</td>
-		<td >10</td>
-	</tr>
-	<tr>
-		<td >Cooling M.T., Rouilly V., Misirli G., Lawson J., Yu T., Hallinan J., Wipat A.</td>
-		<td >Standard virtual biological parts: A repository of modular modeling components for synthetic biology</td>
-		<td >2010</td>
-		<td >Bioinformatics</td>
-		<td >10</td>
-	</tr>
-	<tr>
-		<td >Brown J.</td>
-		<td >The iGEM competition: Building with biology</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >7</td>
-	</tr>
-	<tr>
-		<td >Bikard D., Kepes F.</td>
-		<td >First French team success during iGEM Synthetic biology competition [Succ√É¬®s de la premi√É¬®re √É¬©quipe fran√É¬ßaise lors de la comp√É¬©tition iGEM de biologie synth√É¬©tique]</td>
-		<td >2008</td>
-		<td >Medecine/Sciences</td>
-		<td >6</td>
-	</tr>
-	<tr>
-		<td >Gallagher R.</td>
-		<td >An iGEM of an idea</td>
-		<td >2007</td>
-		<td >Scientist</td>
-		<td >2</td>
-	</tr>
-	<tr>
-		<td >Haynes K.A., Broderick M.L., Brown A.D., Butner T.L., Harden L., Heard L., Jessen E., Malloy K., Ogden B., Rosemond S., Simpson S., Zwack E., Campbell A.M., Eckdahl T., Heyer L.J., Poet J.L.</td>
-		<td >Computing with living hardware</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >2</td>
-	</tr>
-	<tr>
-		<td >Lohmueller J., Neretti N., Hickey B., Kaka A., Gao A., Lemon J., Lattanzi V., Goldstein P., Tam L.-K., Schmidt M., Brodsky A.S., Haberstroh K., Morgan J., Palmore T., Wessel G., Jaklenec A., Urabe H., Gagnon J., Cumbers J.</td>
-		<td >Progress toward construction and modelling of a tri-stable toggle switch in E. coli</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >2</td>
-	</tr>
-	<tr>
-		<td >Hafner S.</td>
-		<td >iGEM competition 2008, two French teams! [Comp√É¬©tition iGEM 2008: Deux √É¬©quipes fran√É¬ßaises!]</td>
-		<td >2009</td>
-		<td >Medecine/Sciences</td>
-		<td >2</td>
-	</tr>
-	<tr>
-		<td >Hafner S.</td>
-		<td >IGEM 2009: Synthethic biology and ethics [Les iGEM 2009 la biologie synth-√É¬©thique]</td>
-		<td >2010</td>
-		<td >Medecine/Sciences</td>
-		<td >2</td>
-	</tr>
-	<tr>
-		<td >Dabholkar S., Thattai M.</td>
-		<td >Brainstorming biology</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >1</td>
-	</tr>
-	<tr>
-		<td >Brown T., Chang C., Heinze B., Hollinger P., Kittleson J., MacDow K., Reavis D., Curry J., Riley M.</td>
-		<td >Development of an inducible three colour bacterial water colour system</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >1</td>
-	</tr>
-	<tr>
-		<td >Rodrigo G., Montagud A., Aparici A., Aroca M.C., Baguena M., Carrera J., Edo C., Fernandez-De-Cordoba P., Ferrando A., Fuertes G., Gimenez D., Mata C., Medrano J.V., Navarrete C., Navarro E., Salgado J., Tortosa P., Urchueguia J., Jaramillo A.</td>
-		<td >Vanillin cell sensor</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >1</td>
-	</tr>
-	<tr>
-		<td >Badalamenti J.P., Weiss L.E., Buckno C.J., Richard T.L., Weiss P.S., Cirino P.C.</td>
-		<td >Synthetic sports: A bacterial relay race</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >1</td>
-	</tr>
-	<tr>
-		<td >Dixon J., Kuldell N.</td>
-		<td >BioBuilding: Using banana-scented bacteria to teach synthetic biology</td>
-		<td >2011</td>
-		<td >Methods in Enzymology</td>
-		<td >1</td>
-	</tr>
-	<tr>
-		<td >Quiroz J.N.A., Flores R.B., Cisneros T.G.B., Rosales I.Y.F., Naranjo A.G., Sanchez J.C.G., Jimenez M.E.G., Padilla R.E.G., Baena A.J.L., Hernandez P.A.L., Padilla P.G., Miller R.P., Gaspar I.N.R., Chico J.C.R., Martinez A.R., Romero J.P., Arzate A.S., Barradas J.S.A., Diaz D.A., Bracho A.B., Benitez C., Arteag C.I.F., Quiroz F.H., Martinez G.J., Rabadan J.L., Salvador M.C.O., Longoria P.P., Orozco R.P., Corona F.R., Majarrez E.S., Hassan E.S., Sanchez C.S., Saldana U.V., Segura P.B.Z.</td>
-		<td >Biological implementation of algorithms and unconventional computing</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >0</td>
-	</tr>
-	<tr>
-		<td >King P., Lavrovsky V., Von Mammen S., Jacob C.</td>
-		<td >Teaching bacteria how to dance</td>
-		<td >2007</td>
-		<td >IET Synthetic Biology</td>
-		<td >0</td>
-	</tr>
-	<tr>
-		<td >Edwards C.</td>
-		<td >The gene machines</td>
-		<td >2008</td>
-		<td >Engineering and Technology</td>
-		<td >0</td>
-	</tr>
-	<tr>
-		<td >Bachman R.</td>
-		<td >An iGEM of an idea? [2]</td>
-		<td >2008</td>
-		<td >Scientist</td>
-		<td >0</td>
-	</tr>
-	<tr>
-		<td >Kaczkowski P.</td>
-		<td >An iGEM of an idea? [1]</td>
-		<td >2008</td>
-		<td >Scientist</td>
-		<td >0</td>
-	</tr>
-	<tr>
-		<td >Jerala R.</td>
-		<td >I like iGEM</td>
-		<td >2009</td>
-		<td >Scientist</td>
-		<td >0</td>
-	</tr>
-	<tr>
-		<td >Dress L.</td>
-		<td >iGEM 2010: Synthetic biologists compete for the future</td>
-		<td >2010</td>
-		<td >Industrial Biotechnology</td>
-		<td >0</td>
-	</tr>
-	<tr>
-		<td >Gu X., Trybilo M., Ramsay S., Jensen M., Fulton R., Rosser S., Gilbert D.</td>
-		<td >Engineering a novel self-powering electrochemical biosensor</td>
-		<td >2010</td>
-		<td >Systems and Synthetic Biology</td>
-		<td >0</td>
-	</tr>
-	<tr>
-		<td >Stewart D., Wilson-Kanamori J.R.</td>
-		<td >Modular modelling in synthetic biology: Light-based communication in E. coli</td>
-		<td >2011</td>
-		<td >Electronic Notes in Theoretical Computer Science</td>
-		<td >0</td>
-	</tr>
-	<tr>
-		<td >Mitchell R., Dori Y.J., Kuldell N.H.</td>
-		<td >Experiential Engineering Through iGEM-An Undergraduate Summer Competition in Synthetic Biology</td>
-		<td >2011</td>
-		<td >Journal of Science Education and Technology</td>
-		<td >0</td>
-	</tr>
-	<tr>
-		<td >Boyle P.M., Burrill D.R., Inniss M.C., Agapakis C.M., Deardon A., DeWerd J.G., Gedeon M.A., Quinn J.Y., Paull M.L., Raman A.M., Theilmann M.R., Wang L., Winn J.C., Medvedik O., Schellenberg K., Haynes K.A., Viel A., Brenner T.J., Church G.M., Shah J.V., Silver P.A.</td>
-		<td >A BioBrick compatible strategy for genetic modification of plants</td>
-		<td >2012</td>
-		<td >Journal of Biological Engineering</td>
-		<td >0</td>
-	</tr>
-	<tr>
-		<td >Hesselman M.C., Odoni D.I., Ryback B.M., de Groot S., van Heck R.G.A., Keijsers J., Kolkman P., Nieuwenhuijse D., van Nuland Y.M., Sebus E., Spee R., de Vries H., Wapenaar M.T., Ingham C.J., Schroen K., Martins dos Santos V.A.P., Spaans S.K., Hugenholtz F., van Passel M.W.J.</td>
-		<td >A multi-platform flow device for microbial (co-) cultivation and microscopic analysis</td>
-		<td >2012</td>
-		<td >PLoS ONE</td>
-		<td >0</td>
-	</tr>
-	<tr>
-		<td >Materi W.</td>
-		<td >Leading a successful iGEM team</td>
-		<td >2012</td>
-		<td >Methods in Molecular Biology</td>
-		<td >0</td>
-	</tr>
-	<tr>
-		<td >Ho-Shing O., Lau K.H., Vernon W., Eckdahl T.T., Campbell A.M.</td>
-		<td >Assembly of standardized DNA parts using biobrick ends in E. coli</td>
-		<td >2012</td>
-		<td >Methods in Molecular Biology</td>
-		<td >0</td>
-	</tr>
-	<tr>
-		<td >Giavitto J.-L.</td>
-		<td >The modeling and the simulation of the fluid machines of synthetic biology</td>
-		<td >2012</td>
-		<td >Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)</td>
-		<td >0</td>
-	</tr>
-	<tr>
-		<td >Muller K.M., Arndt K.M.</td>
-		<td >Standardization in synthetic biology</td>
-		<td >2012</td>
-		<td >Methods in Molecular Biology</td>
-		<td >0</td>
-	</tr>
-	<tr>
-		<td >Kuldell N.</td>
-		<td >Living machines: Some assembly required: Kit-based competitions challenge teams of students to learn microbiology and design principles in the context of synthetic biology</td>
-		<td >2012</td>
-		<td >Microbe</td>
-		<td >0</td>
-	</tr>
-</table>
-                </div>
-              </div>
-            </div>
-          </div>
+<div class="row">
+<div class="span5">
-</section> <!-- End "old-data" -->
+<h3>Expectations</h3>
+<p>One of the iGEM competition's important goals is to build up well-characterized BioBrick content in the Registry. Thus, the iGEM competition and the Registry are inherently linked. Hence we would expect publications that mention iGEM to also refer to the Registry. However, as the Registry is not <em>only</em> used by iGEM, we expected to also find a large number of papers that mention the Registry <em>without</em> mentioning iGEM.</p>
+<p>In other words, we expected:
+<ul>
+<li>a <em>large proportion</em> of papers that mention <em>iGEM and the Registry</em>,</li>
+<li>a <em>large proportion</em> that mention <em>only the Registry</em> and
+<li>only a <em>very small proportion</em> that mention <em>only iGEM</em>.</li>
+</ul>
+</p>
+<h3>Yes and No</h3>
+<p>The expected proportion of papers mentioning <em>only the Registry</em> was found. However, of the papers that mention iGEM, about half do not mention the Registry. This was a greater proportion than we expected.</p>
+<p>We theorise that this may be due to the iGEM Foundation and iGEM teams <em>under-emphasizing</em> their importance to part standardisation. Indeed iGEM is usually presented by the Foundation and teams as "a synthetic biology competition", when really that's just half the picture. We're not just competing, but we also exist for a greater good: <em>Make the Registry better</em> and do our part in helping organise synthetic biology!</p>
+</div>
+<div class="span7">
+<a href="#modal-proportional-mentions" data-toggle="modal" class="thumbnail">
+<img src="https://static.igem.org/mediawiki/2012/1/15/DataIgemVsRegistry.png" />
+    <p class="label-set">
+    <span class="label" style="background-color:#b0db54">Just iGEM</span>
+    <span class="label" style="background-color:#ff7900">iGEM and Parts Registry</span>
+    <span class="label" style="background-color:#81ccf1">Just Parts Registry</span>
+    </p>
+<h5>Chart 5: <em>Proportional mentions of iGEM, Registry and Both</em></h5>
+<p>This chart is a <strong>stacked</strong> combination of the three charts above. It shows how often <em>iGEM</em>, <em>the Registry</em> and <em>both</em> are mentioned in scientific articles (i.e. how many papers from each year match our search terms). Each of the three quantities has stayed <em>roughly equal</em> to the others throughout the years.</p>
+</a><!-- /thumbnail -->
+</div>
+</div><!-- Close row -->
+</section><!-- Close data analysis -->
+<section id="citations">
+<div style="background-image:url('https://static.igem.org/mediawiki/2012/6/61/Papers_colour.jpg');background-position:top right;background-repeat:no-repeat;background-size:30%" />
+<div class="row">
+<div class="span9">
+<div class="page-header">
+	<h1>Citations <small>How is iGEM cited and what can we learn from it?</small></h1>
+</div>
+<div class="row-fluid">
+<div class="span6">
+<h2>Sourcing data</h2>
+<p>In order to analyse how exactly iGEM and the Registry are being cited, we decided to manually examine a set of the papers in our results. We had discarded Scopus and Web of Knowledge earlier when carrying out wide-range data collection. However, for this focused search, the small but certain selection of papers that Scopus and Web of Knowledge gave us was perfect!</p>
+<p>We manually combined the results obtained from Scopus and Web of Knowledge. We then deleted duplicates and the few remaining irrelevant publications.</p>
+<p>The keyword “iGEM” gave <strong>41</strong> publications combined from Scopus and <acronym title="Web of Knowledge">WoK</acronym>. Of these, we discarded 16 publications for various reasons:
+<ul>
+<li>5 texts were non-research articles, such as magazine articles</li>
+<li>2 were articles on bioethics that mentioned iGEM only in passing</li>
+<li>1 article cited an articles with “iGEM” in the title but was itself unrelated</li>
+<li>6 were articles we couldn’t locate or access despite the University of St Andrews having subscriptions to various publishers</li>
+<li>2 were in French.</li>
+</ul>
+</p>
+<p>This left <strong>25</strong> articles.</p>
+</div>
+<div class="span6">
+<h2>Discussion</h2>
+<p>We considered citations <em>sufficient</em> when they cited the Registry or named the specific BioBricks used (when appropriate) <em>and</em> citing either the Knight or Endy paper about BioBrick assembly.</p>
+<p>We examined those 25 articles for how they cited iGEM and the Registry. We found that <strong>11</strong> articles cited iGEM and/or the Registry of Standard Biological Parts satisfactorily.</p>
+<p>There were two common forms of citation:</p>
+<p>
+<ul>
+<li><em>Registry of Standard Biological Parts [http://www.partsregistry.org].</em></li>
+<li><em>Knight, T. F. (2003). Idempotent Vector Design for Standard Assembly of Biobricks.<br />
+DOI: 1721.1/21168.</em></li>
+</ul>
+</p>
+<p>5 articles did not cite the Registry or BioBricks sufficiently according to our demands.  For one of them, we question its content relevancy. The remaining 4 show examples of simple in-text citation of the Registry, with or without a hyperlink, no mention of the specific BioBrick used, or no mention of BioBricks at all. Surprisingly, each of these papers was associated with an iGEM team.</p>
+<p>There are 9 additional papers we found in the journal IELTS Synthetic Biology, all of which show vast variations in content, research, and citation quality. The journal seems to have been published once, and seems to have asked all 2006 iGEM finalists to submit a paper based on their research. This resulted in some sub-par articles. Because they weren't peer-reviewed and due to their strong biasing affiliation with iGEM, we disregarded them from our overall data set.</p>
+</div>
+</div>
+</div>
+<div class="span3"></div>
+</div>
+<div class="well">
+<h2>Our recommendations</h2>
+<p>Reviewing the data, we have concluded that there are standard methods of citation being used by the scientific community to refer to the Parts Registry. In order for the Registry to uphold <em>referencing standards</em> as well as Parts standards within synthetic biology, we think this method of referencing should be officially recommended on the Registry and iGEM website.</p>
+<p>A clear and standard citation method would support teams who are attempting to publish and let them set an example of citation style to the rest of the scientific community. Additionally, clearly stating a standard method of citation would make citing of the Registry easier and so motivate its citation in general. Of course, more citations means more attention and adoption within the field of synthetic biology.</p>
+<p>Teams participating in iGEM should be encouraged to cite properly and to try to publish their work. A tutorial of some kind hosted on the iGEM website would help. Such things (among many other bright ideas) were proposed in 2008  by <a href="http://openwetware.org/wiki/User:Macowell/Making_iGEM_Better">Cowell</a>, but haven't been implemented. We see a standardised citation method as a high priority for maximising iGEM's scientific influence.</p>
+</div>
+</section>
+<section id="conclusion">
+<div class="page-header">
+	<h1>Conclusion <small>Our <em>Human Practices</em> in a nutshell...</small></h1>
+</div>
+<p>We found that <span style="background-color:#ecf7d5"><strong>the iGEM competition is making a positive impact.</strong></span> The competition is growing in size and scope, and both iGEM and the Registry are netting a proportionally increasing amount of attention from the scientific community. We are doing well!</p>
+<p>However, we also found that <span style="background-color:#d1edf9"><strong>quite a number of discussions of iGEM <em>miss</em> the important connection between our iGEM competition and the Parts Registry</strong></span>. We recommend that the iGEM Foundation and future teams emphasise the iGEM competition's <abbr title="reason for existence">raison d'être</abbr> clearly in the future.</p>
+<p>We also noticed that <span style="background-color:#ffe6cd"><strong>some papers do not give sufficient or clear credit to the Registry.</strong></span> We interpret this as confusion as to how the Registry should be cited. We recommend that a standardised referencing should be introduced. This would support the publishing process for inexperienced undergraduates involved in the competition, gather greater attention for the iGEM Foundation and Parts Registry and hence further our aims of synthetic biology standardisation. ∎</p>
+</section>
-<!-- testing 123
+<!-- References
 ================================================== -->
-<section id="nonsense-graphs">
+<section id="references">
-	<br>
+<div class="page-header">
-	<h1>Graphs (currently nonsense test data)</h1>
+	<h1>References</h1>
-<div id="chart_div2" style="width: 900px; height: 500px;"></div>
+</div>
-<p>hi</p>
-<div id="chart_div" style="width: 900px; height: 500px;"></div>
+<p><strong>"Ael 2" and "Vulpecula", 2012.</strong> h-index (Hirsch). <i>Wikipedia</i>. [image online] Available at: &lt;<a href="http://en.wikipedia.org/wiki/File:H-index-en.svg">http://en.wikipedia.org/wiki/File:H-index-en.svg</a>&gt; [Accessed Jul 27, 2012].</p>
-</section><!-- End "data-analysis" -->
+<p><strong>"Ael 2", 2012.</strong> Illustrated example for the g-index proposed by Egghe. <i>Wikipedia</i> [image online]  Available at: &lt;<a href="http://en.wikipedia.org/wiki/File:Gindex1.jpg">http://en.wikipedia.org/wiki/File:Gindex1.jpg</a>&gt; [Accessed Jul 27, 2012].</p>
+<p><strong>Cowell, M.L., 2008.</strong> Making iGEM Better. [web page] Available at &lt;<a href="http://openwetware.org/wiki/User:Macowell/Making_iGEM_Better">http://openwetware.org/wiki/User:Macowell/Making_iGEM_Better</a>&gt; [Accessed Jul 19, 2012].</p>
+<p><strong>Egghe, L., 2006.</strong> Theory and practise of the g-index. <i>Scientometrics</i> [online], Volume 69 (Issue 1), p.131-152. Available at: &lt;<a href="www.springerlink.com/content/4119257t25h0852w/?MUD=MP">www.springerlink.com/content/4119257t25h0852w/?MUD=MP</a>&gt; [Accessed Jun 7, 2012].</p>
+<p><strong>Harzing, A.W., 2007.</strong> Publish or Perish. [computer program] Available from &lt;<a href="http://www.harzing.com/pop.htm">http://www.harzing.com/pop.htm</a>&gt;</p>
+<p><strong>Hirsch, J.E., 2005.</strong> An index to quantify an individual's scientific research output. <i>Proceedings of the National Academy of Sciences of the United States of America</i>, Volume 102 (Issue 46). [online] Available at: http://&lt;<a href="www.ncbi.nlm.nih.gov/pmc/articles/PMC1283832/?tool=pmcentrez">www.ncbi.nlm.nih.gov/pmc/articles/PMC1283832/?tool=pmcentrez</a>&gt; [Accessed 5th Jul, 2012]</p>
+<p><strong>iGEM Foundation, 2012a.</strong> Previous iGEM Competitions. [web page] Available at: &lt;<a href="https://igem.org/Previous_iGEM_Competitions">https://igem.org/Previous_iGEM_Competitions</a>&gt; [Accessed Jul 30, 2012]</p>
+<p><strong>iGEM Foundation, 2012b.</strong> Press Kit. [web page] Available at: &lt;<a href="https://igem.org/Press_Kit">https://igem.org/Press_Kit</a>&gt; [Accessed Aug 3, 2012]</p>
+<p><strong>Iselid, L., 2006.</strong> Research on citation search in Web of Science, Scopus and Google Scholar. <i>One Entry to Research</i> [blog] Available at: &lt;<a href="http://oneentry.wordpress.com/2006/08/11/research-on-citation-search-in-web-of-science-scopus-and-google-scholar/">http://oneentry.wordpress.com/2006/08/11/research-on-citation-search-in-web-of-science-scopus-and-google-scholar/</a>&gt; [Accessed Jun 20, 2012].</p>
+<p><strong>Péter J., 2006.</strong> Dubious hit counts and cuckoo's eggs. <i>Online Information Review</i> [online] Volume 30 (Issue 2) p.188-193. Available at: &lt;<a href="http://www.emeraldinsight.com/journals.htm?articleid=1550726&show=abstract">http://www.emeraldinsight.com/journals.htm?articleid=1550726&show=abstract</a>&gt; [Accessed Jun 20, 2012].</p>
+<p><strong>Zhou  Y., Liyan L. and Menghui L., 2012.</strong> Quantifying the influence of scientists and their publications: distinguishing between prestige and popularity. <i>New Journal of Physics</i>, [online] Volume 14 (March 2012) Available at: &lt;<a href="http://iopscience.iop.org/1367-2630/14/3/033033/">http://iopscience.iop.org/1367-2630/14/3/033033/</a>&gt; [Accessed Jun 7, 2012].</p>
+</section><!-- Close references -->
+</div><!-- /container -->
+<!-- Chart modals -->
+<div class="modal large hide" id="modal-attention-summary">
+  <div class="modal-header">
+    <button type="button" class="close" data-dismiss="modal">×</button>
+    <h3>Chart zoom: <em>iGEM success summary</em></h3>
+  </div>
+  <div class="modal-body">
+    <img src="https://static.igem.org/mediawiki/2012/a/af/DataSummaryStats.png" alt="">
+  </div>
+</div>
+<div class="modal large hide" id="modal-igem-attention">
+  <div class="modal-header">
+    <button type="button" class="close" data-dismiss="modal">×</button>
+    <h3>Chart zoom: <em>iGEM attention</em></h3>
+  </div>
+  <div class="modal-body">
+    <img src="https://static.igem.org/mediawiki/2012/1/12/Data1Chart_ignoresBefore2k.png" alt="">
+  </div>
+</div>
+<div class="modal large hide" id="modal-igem-registry-attention">
+  <div class="modal-header">
+    <button type="button" class="close" data-dismiss="modal">×</button>
+    <h3>Chart zoom: <em>iGEM and Registry attention</em></h3>
+  </div>
+  <div class="modal-body">
+    <img src="https://static.igem.org/mediawiki/2012/e/ee/Data6Chart.png" alt="">
+  </div>
+</div>
+<div class="modal large hide" id="modal-registry-attention">
+  <div class="modal-header">
+    <button type="button" class="close" data-dismiss="modal">×</button>
+    <h3>Chart zoom: <em>Registry attention</em></h3>
+  </div>
+  <div class="modal-body">
+    <img src="https://static.igem.org/mediawiki/2012/3/3f/Data3Chart.png" alt="">
+  </div>
+</div>
+<div class="modal large hide" id="modal-proportional-mentions">
+  <div class="modal-header">
+    <button type="button" class="close" data-dismiss="modal">×</button>
+    <h3>Chart zoom: <em>Proportional mentions</em></h3>
+  </div>
+  <div class="modal-body">
+    <img src="https://static.igem.org/mediawiki/2012/1/15/DataIgemVsRegistry.png" alt="">
+  </div>
+</div>
-    </div><!-- /container -->
+<!-- End of chart modals -->
-  </body>
+</body>
 </html>
 {{:Team:St_Andrews/Template:Footer}}

Team:St Andrews/Human-practices

From 2012.igem.org

Latest revision as of 20:00, 26 September 2012

Introduction

Data collection Data and where to find it

Query summary

Why we used Google Scholar

Browse the data

On extraction tools

Metrics Quantifying scientific impact

Plain citation count

h-index

g-index

Algorithmic methods

Data analysis What does this data actually mean?

Is our data usable?

Yes. Mostly...

...but only mostly.

What's is iGEM's impact?

Chart 1: Summary of iGEM-related attention over time

And the answer is...

Where this data comes from

Is the relationship between iGEM and the Registry clear?

Chart 2: "iGEM" attention

Chart 3: "Registry" attention

Chart 4: "iGEM and Registry" attention

Expectations

Yes and No

Chart 5: Proportional mentions of iGEM, Registry and Both

Citations How is iGEM cited and what can we learn from it?

Sourcing data

Discussion

Our recommendations

Conclusion Our Human Practices in a nutshell...

References

Chart zoom: iGEM success summary

Chart zoom: iGEM attention

Chart zoom: iGEM and Registry attention

Chart zoom: Registry attention

Chart zoom: Proportional mentions