Public engagement

Public engagement or public participation is a term that has recently been used to describe "the practice of involving members of the public in the agenda-setting, decision-making, and policy-forming activities of organizations/institutions responsible for policy development."^[1] It is focused on the participatory actions of the public to aid in policy making based in their values.^[2][1]

Origins

The tradition of a decision-making body getting inputs from those with less power is generally known as "consultation". This became popular with UK governments during the 1980s and 1990s. Even though most governments that carry out consultations are democratically elected, many people who became involved in these processes were surprised that conduct of such "consultations" was unsatisfactory in at least three respects.

1. Groups that already had influence were often the only ones consulted
2. People who did not have the resources would usually not be able to be part of a consultation, even if the decision it was meant to influence might have a major impact on them.
3. There were no agreed safeguards against consultations being used cynically by decision-makers to make it look like they had sought to canvass other opinions, while in fact having set a new policy in place even before it asked the question.

As early as 1979, science analyst Dorothy Nelkin pointed out that much of what passed for participation in governance could best be understood as attempts by the powerful to co-opt the public.

Rationale

Public engagement is a relatively new term, hardly used before the late 1990s. The existing term it shares most in common with is participatory democracy, discussed by thinkers such as Jean-Jacques Rousseau, John Stuart Mill and G D H Cole.

Many see participatory democracy as complementing representative democratic systems, in that it puts decision-making powers more directly in the hands of ordinary people. Rousseau suggested that participatory approaches to democracy had the advantage of demonstrating that "no citizen is a master of another" and that, in society, "all of us are equally dependent on our fellow citizens". Rousseau suggested that participation in decision – making increases feeling among individual citizens that they belong in their community. Perhaps the most long-standing institution of participatory democracy is the system of trial by jury.

Whilst elected governments make the laws, it is therefore juries that are able to decide the innocence or guilt of anyone charged with breaking many of those laws, making it a key instrument of participatory democracy. Over the centuries they have achieved an importance to many democracies that have had to be fiercely defended. One senior judge surveying the limiting of a government's power provided by the jury over the centuries compared the jury to: "a little parliament... No tyrant could afford to leave a subject's freedom in the hands of twelve of his countrymen.... Trial by jury is more than an instrument of justice and more than one wheel of the constitution: it is the lamp that shows that freedom lives". (Patrick Devlin 1956). Today, jury trials are practised in many democracies around the world including the US, UK, Russia, Spain, Brazil and Australia. Perhaps no other institution of government rivals the jury in placing power so directly in the hands of citizens, or wagers more on the truth of democracy's core claim that the people make their own best governors. Juries are therefore argued to be the most widespread form of genuine consultation at work in society today.

The tension between the state and civil society as underscored by Public Engagement within Newly Industrialized Economies (NIE) such as Singapore is illustrated by Kenneth Paul Tan of the Lee Kuan Yew School of Public Policy^[3]

But speaking about public engagement is, of course, quite a different thing from carrying out public engagement. And this is where there seems to be a gap between rhetoric and practice in Singapore. For instance, government officials recently met selectively with concerned members of the public to discuss a controversial decision to build a road through a historically significant graveyard. When criticised for not taking the public's views seriously, the Government explained that the meeting was never meant to be a "consultation". So it is important to ask why such a gap exists and why it might be difficult to close it, assuming of course that closing it is what we want to do."

As a neo-liberal global city, Singapore has been witnessing rising popular pressure. Politics has come to the fore again, prompting the policy establishment to pay greater heed to the demands of a new and more variegated citizenry, with political leaders now more sensitive to the real prospect of losing elections. At the same time, the cultural, ideological, practical and institutional legacies of the earlier survivalist and development stages continue to be a source of tension in the evolution of Singapore's political culture. By no means has this been a simple and linear story of liberalisation. However, are these recent developments enough to shift the deeply entrenched public sector mindsets that have been formed out of historically shaped ways of thinking and reasoning? Will a new generation of leaders in the public sector, whose horizons of experience may differ from the survivalist and developmental preoccupations of a previous generation, lead to fresh opportunities for new terms of engagement?

The elitist proclivities of the public sector, reinforced by top-level salaries that are comparable to the private sector, are unlikely to incentivise real public engagement, since they reinforce the sense that public sector leaders, possessing superior intellect, knowledge and insight, must defend the public interest against irrational and dangerous mass populism. The public, according to this mindset, needs to be educated to think correctly rather than present themselves as equal participants in policy formulation and implementation.

Modalities of engagement

It is important to note that the participation of the public can occur at many different levels. Due to this ambiguity, modalities for public engagement have been categorized based on the types of information flow and level of involvement of the public and/or sponsor (i.e. academia, government, private sector):^[1][4][5]

Public communication is characterized by the top-down, one-way transfer of information or resources from initiators of an engagement, like government agencies to the public and where feedback from the public is not returned. This includes mechanisms like information broadcasts, static website resources, newsletters, public service announcements, or informational outreach through the legacy and social media. Traditional media functions in this way by influencing the public agenda, termed agenda-setting theory.

Public consultation is the collection of feedbacks and information about or from the public with the sponsor. Potential mechanisms are referendum, surveys, focus groups, or interactive websites. These mechanisms gather information from the public who are or will be affected by those decisions to shape what sponsors focus on or invest their resources into. Although public consultation involves soliciting public feedbacks, it is still a one-way flow of information, but in the reverse direction of public communication. The initiators of public consultation retain decision-making authority.

Public involvement is interactive and involves the transfer of fact-based information, values, and beliefs between both the public and the sponsor (i.e., typically, experts or policy makers). Deliberative opinion polls serve as a mechanism in which the public's opinions are collected before and after receiving information from the sponsor.

Social media has become an increasingly prominent mechanism of public involvement as well. Social media allows for instant and on-going dialogue between sponsors and the public. Additionally, social is flexible and can be implemented in a variety of ways, but not all social media platforms function identically. For example, three major health organizations implemented social media campaigns during the Ebola epidemic of 2013: Centers for Disease Control and Prevention (CDC), World Health Organization (WHO), and Médecins Sans Frontières (MSF, also known as Doctors without Borders). All three organizations used both Twitter and Instagram to communicate with the public, but the public engaged with Instagram posts more often than Twitter. This finding is consistent with Visual communication theory. Additionally, each organization fostered a different level of engagement with the public (MSF garnered the highest engagement).^[6]

Public collaboration is another form of two-way dialogue, but it also involves collaboration between the public and sponsor where they actively work together to identify and create suitable solutions to the challenges under discussion. Mechanisms for public participation include action planning workshops, citizens' jury, consensus conferences, and task forces.^[1]

Public empowerment is where the decision-making authority is placed majorly, if not solely, on the public in which they are provided with enough information from the sponsor and collectively come to a formal decision. This decision is then binding on the affected public. This is the rarest engagement mechanism, because the sponsors (i.e., policy makers and regulatory actors) are not allowed to transfer their decision-making authority to the public.

Good practice

Taking participatory democracy as an ideal for public engagement has significant consequences for how we apply the concept to issues with a scientific or technical element. Instead of merely receiving inputs from various interested parties, a participatory model of consultation forces decision-makers to recognise the democratic accountability of their actions not merely every few years at elections, but in a more systematic, direct sense to citizens.^[7]

A common misconception is that there is a particular methodology that can be devised to facilitate all public engagement. Effective participation, by contrast, is conducted on the assumption that each different situation will require a different design, using a new combination of tools as part of an evolving cycle of action and reflection by the institution involved.^[8]

Each "experiment" in participatory democracy contains a unique mix of people and institutions. Each method must therefore select elements from a range of different approaches.^[9] Participation is also overtly "political" in that it is about humans, power and knowledge – all of which are inherently complex and which together make for a potent mix that requires sensitivity and careful planning. So while participatory processes can be replicated in the same way as scientific protocols, their human ingredients can differ so much that a concentration on replicating what happened elsewhere often hinders the practical application of a technique. Each consultation event needs to proceed from an understanding of its political, scientific, institutional and practical constraints.

The effectiveness of public engagement methods can be assessed by their fairness and efficiency in achieving its purpose. Fairness addresses whether the public perceives their information was collected by sponsors in a way that equally represents the affected population. Although, scholars suggest that assessing fairness of public engagement is a complexity in itself. Efficiency "refers to maximizing the relevant information (knowledge and/or opinions) from the maximum number of relevant sources and transferring this efficiently to the appropriate receivers."^[1] Much of the effectiveness of public engagement methods rely on how these methods are conducted and if they are at the will of interfering biases or confounding variables.

So instead of recommending a perfect method of public engagement, working principles for such processes based on those used by PEALS at Newcastle University are listed below.

Nine principles

1. Participants should join those organising the process in setting terms of reference for the whole exercise, and framing the questions that they will discuss.
2. The group organising, or in overall control of, the process should be broad based, including stakeholders with different interests on the subject being discussed.
3. There should be a diversity of information sources and perspectives available to participants.
4. There should be space for the perspectives of those participants who lack specialist knowledge of the area concerned to engage in a two-way exchange with those possessing specialist knowledge.
5. There should be complete transparency of the activities carried out within the process to those both inside and outside it.
6. Those without a voice in policy-making should be enabled to use the consultation process as a tool for positive political change. This should be embedded in the process by sufficient funds being made available for follow-up work after their initial conclusions have been reached.
7. The process should contain safeguards against decision-makers using a process to legitimise existing assumptions or policies.
8. All groups involved in the process should be given the opportunity to identify possible strategies for longer-term learning, development and change on a range of issues relating to their conclusions.
9. The group organising, or in overall control of, the process should develop an audit trail through the process, to explain whether policies were changed, what was taken into account, what criteria were applied when weighing up the evidence from the process, and therefore how the views of those involved in the participatory process may have made a difference. This should be explored together with as many those involved in all levels of the process as possible.

In science and technology

Main article: Public engagement with science and technology

The movement for public engagement in science and technology grows out of a paradox: the steadily increasing number of ways citizens can learn about science has not always been matched by increased levels of scientific knowledge or sophistication among citizens. There are nearly one hundred science and technology museums in North America alone, numerous science blogs (the aggregation site, ScienceBlogs, reported 152 thousand posts and 3.3 million comments for 61 blogs alone before it closed in October 2017), and a proliferating number of science magazines.^[10]

However, surveys of scientific literacy show a long term pattern in which Americans have only a moderate understanding of basic scientific facts and concepts. In 1992, only 59% of adults sampled could give correct answers to a series of scientific terms and concepts; in 2008 the number was 64%.^[11] But in 2010, the validity of these measures of scientific literacy became controversial. Americans performed much worse on questions about evolution and the Big Bang theory than respondents from different countries. These differences disappeared when short caveats like, "According to the theory of Evolution..." were added to the questions – pointing to a larger conflict between scientific knowledge and personal beliefs in the U.S.^[12] Another survey found widening gaps in knowledge of nanotechnology between the most and least educated,^[13] highlighting the knowledge gaps that exist between different levels of education and the challenges they present for public engagement with science.

To address this disconnect and complexity, there have been calls for new ways of connecting citizens with science in hopes that citizens can do more than respond passively to choices made by experts, and instead actually contribute to shaping science policy as it is made. This engagement of different publics in the policymaking process happens through the bidirectional flow of information and interaction between the relevant publics and the sponsor of the engagement (e.g., policy makers, experts, scientists, community organizations).

Goals for effective public engagement in science

A recent framework for effective public engagement delineates seven goals for public engagement activities.^[4] While initially designed for emerging science-based technologies like CRISPR, the goals are sufficiently broad to function as an analytical tool, guiding our assessment of the effectiveness of public engagement efforts across various scientific topics. The public engagement, here, is defined as “processes and initiatives focused on enabling public participation in the responsible innovation and development of new technologies, including the management and assessment of technological risk.” ^[4] The goals include:

1. Avoid potential controversy - scientists must instill a sense of trust among the public and be willing to listen to their feedback
2. Educate the public - make information accessible to the public and appeal to "unreasonable objections" while understanding the efficacy of correcting knowledge deficits
3. Build democratic capacity through deliberation - increase citizens’ willingness to reflect with an open mind on others’ views, fostering genuine and respectful interactions across lines of difference
4. Widen the representation of voices - create forums for previously excluded voices and perspectives that would not have emerged if experts had simply deliberated among themselves
5. Solicit input on values-based debates triggered by science - have the public raise ethical, legal, and sociopolitical issues that go beyond the problems identified by the experts
6. Enable responsible innovation - involve the public as early and as often as possible in the development stage to shape the direction of scientific advancements collaboratively
7. Shape public policy - create initiatives where public engagement can have direct influence on policy making

Challenges for scalable public engagement efforts in science

There is no "one-size-fits-all" approach to successful engagement

The challenge for scalable public engagement exercises lies in the absence of a “one-size-fits-all” modality of engagement that can be deployed across intended outcomes and contexts.^[4] Many engagement efforts are therefore employed simultaneously with overlapping goals, as no single engagement method can effectively cater the unique needs of different publics (with varied values, beliefs, socioeconomic circumstances, and risk perceptions) and the specific context of the issue at hand. The modalities employed by sponsors of public engagement initiatives may vary across at least five different dimensions: 1) intended outcomes, 2) (the stage of) the issue/controversy, 3) social and policy contexts, 4) intended participants/stakeholders, and 5) resources available.^[4] Thus, designing effective public engagement should involve careful adjustment along these five (or more) dimensions.

Knowledge deficit model of thinking

The public engagement to increase science literacy or change policy in line with what researchers think is best for the public as the ultimate goal of communication represents a knowledge-deficit model of thinking.^[14] Surveys of scientific association members suggest that this "knowledge deficit thinking" persists within the scientific community, with many members prioritizing communication aimed at defending science from misinformation and educating the public about science.^[15] However, decades of research on heuristic decision-making demonstrate that merely correcting individuals' "knowledge deficits" may not be sufficient.^[16] Education has the potential to trigger motivated reasoning, leading to even greater polarization on certain subjects.^[4] This has been observed in individuals' beliefs, attitudes, and behaviors concerning issues such as climate change, the Big Bang, and human evolution.^[17] Additionally, efforts focused on rectifying knowledge deficits can have unintended consequences, such as the "backfire" effect,^[18][19] which occurs when individuals receive scientific information that contradicts their prior beliefs, prompting them to reinforce their existing beliefs regardless of the accuracy.^[20]

Lack of incentive systems and infrastructures related to public engagement for scientists

Another challenge to effective public engagement in science lies in the incentive systems and infrastructures for scientists within academia, government, and the private sector. A 2020 survey-based study discovered that researchers in the U.S. and Canada generally support various public engagement goals, such as ensuring policymakers utilize scientific evidence, promoting a culture that values science, securing adequate research funding, helping people make better individual decisions using science, and fulfilling their duty to society.^[21] Furthermore, a 2020 census of over 6,000 researchers from 46 universities across the U.S. revealed that the majority of science faculty participate in at least one science communication activity (98.3%) and believe public engagement is important (53.2%).^[22] However, even though most scientists considered public engagement crucial, the majority did not think their colleagues or department chairs found engagement activities highly important, nor did they believe residents in their state valued them.^[22] A focus group study involving 23 tenure-track science faculty members from a midwestern U.S. land-grant university in 2020 reported similar findings.^[14] Many scholars identified barriers to conducting public engagement activities, such as feeling pressure to prioritize research and teaching over public engagement due to the lack of emphasis on public service in tenure and promotion requirements. Faculty members also expressed concerns about being perceived as ideological or facing backlash for posting content that could generate criticism from within the university.^[14] These issues highlight the need for practical insights into how institutions can actively incentivize (rather than discourage) participation in engagement activities.

Deliberation is difficult to achieve

Although democratic deliberation can be an effective form of public engagement, some have suggested that these “contexts and styles of interaction are often difficult to produce and to facilitate” and they often fail to scale up. Especially on scientific topics, much public discussion takes place on platforms such as social media, which are inherently limited in their democratic and inclusive capacities.^[4]

In addition, a Pew Research Center report from 2018 reported that an increasing number of Americans find it stressful just to discuss politics with those they disagree with.^[23] This has implications for public deliberation of science in an age when an increasing number of scientific issues, such as COVID-19 or climate change, are entangled with political affiliations.^[24]

Public engagement with science was formally called for in the Third Report of the UK House of Lords Committee on Science and Technology, which argued that "public confidence in science and policy based on science has been eroded in recent year....there is a new humility on the part of science in the face of public attitudes, and a new assertiveness on the part of the public." One consultation, on the regulation of biotechnology in 1998, involved six two-day workshops as well as a large-scale survey. Asked who should be involved in regulating biotechnology, between 40 and 50 percent of respondents said regulatory groups should include a mixed advisory body, an expert body, scientists themselves, the general public, government, and environmental groups. One advisor to the Office of Science and Technology said the process was time-consuming and expensive, and workshops were open to the charge of being run by their organizers rather than their participants, but he still felt the participants dealt with the issues and came to understand them.^[25]

Integrating meaningful public engagement into decision-making is challenging

One of the main challenges in public engagement is ensuring that the input provided by the public is meaningfully incorporated into formal policy-making. Ideally, there should be a feedback loop from various public engagement modalities to the decisions made by legislative bodies or other policy-making institutions. However, in reality, this is often not the case, particularly in the United States, where many federal agencies are legally limited in the extent to which they can offer the public formal decision-making opportunities.^[4] Within the constraints of these legal frameworks, advisory bodies have traditionally enabled "passive" forms of engagement, in which publics and stakeholders can only observe and offer public comments during meetings.^[4] Some have argued that many processes designed for public engagement do not allow the public to say “no” to emerging research or technologies. For example, bioethics commissions established in the 1960s by Congress were supposedly designed to mediate engagement between scientists, lawmakers and the public. However, such commissions have been criticized for have weak democratic accountability and not representing the public's views, even if they adopt an approach based on moral principles (principlism).^[2]

Examples of public engagement

• Americans Discuss Social Security, which engaged 50,000 Americans in all fifty states over fifteen months in 1998 and 1999. During that period, President Clinton and 120 members of Congress took part in town meetings and teleconferences. The project's sponsors state that it "demonstrated the intense public interest in the future of Social Security reform and showed that Americans had more of a "middle ground" approach than special interests or lawmakers had believed. For example, contrary to insiders' expectations, participants overwhelmingly supported raising the cap on payroll taxes."^[26]
• Listening To The City, which brought 5,000 people from New York City and the tri-state area to the Javits Center in July 2002, to discuss the future of lower Manhattan. A separate series of 26 online dialogues involved an additional 800 participants over two weeks. The purpose was to insure broad participation in redevelopment of the World Trade Center site and listen to citizens' ideas about the proposed memorial. According to the final report, 80% of participants were satisfied or very satisfied with the result.^[27]
• Voices & Choices, a 16-county civic engagement process aimed at involving citizens in the economic future of northeast Ohio in 2008. Major agenda items included school funding, government fragmentation and inefficiency, racial isolation and inequalities and creating a competitive workforce.^[28] The resulting meetings engaged 21,000 people. This included one-on-one interviews with three thousand people as well as eleven workshops attended by 15,000 more people. Nine hundred citizens took part in an additional town meeting aimed at identifying goals the region needed to adopt to overcome its most significant challenges. There were also leadership workshops for 1000 government and business leaders. The three top goals that emerged were planning for the future development and growth of the region, ensuring students have the financial resources they need to succeed, and improving workforce training programs. 90% of regional town meeting participants described them as excellent or good. (www.futurefundneo.org/~/media/Final_VoicesChoices_Report.ashx)
• Danish Consensus Conferences, the first of which was originally held in 1987^[29] and organized by the Danish Technology Board, explicitly designed to inform public policy. These conferences have subsequently been held in countries other than Denmark but have shown limited success, with majorities of policymakers reporting to have usually not read the resulting reports.^[4]
• Events in museums and public spaces can be used to successfully communicate with the public ^[30]
• Events within higher education institutions such as universities can also be impactful ^[31]
• Citizen science can be described as "public participation in scientific research," participatory monitoring, and participatory action research whose outcomes are often advancements in scientific research by improving the scientific community's capacity, as well as increasing the public's understanding of science.^[32][33][34] Citizen science has proliferated in the last decade, becoming a critical form of public engagement in science and an increasingly important research tool for the study of large-scale patterns in nature.^[35]

Constraints of public meeting efforts

The following intrinsic and extrinsic constraints of public meetings can lead to unexpected a misrepresentation of the overall public's opinions:

1. Attendance in public meetings is low and highly selective

Although citizens express their intention to participate in public engagement activities, in real world, they are less likely to show up. For example, the average turnout at annual town meetings in Massachusetts in 1996 was 7.6 percent which was much lower than the average municipal election turnout of 31.1 percent.^[36] Low turnout rate in public meetings can lead serious sampling biases when attendees and non-attendees significantly differ in their interests. For example, attendees can be more interested in politics and involved in more personal discourses than non-attendees. In this case, their opinions can be slanted to one side.

2. Group dynamics and personality traits of participants

Depending the makeup of participants, group dynamics and personality characteristics of participants can considerably affect the outcomes of discussions. A small number of outspoken participants can make more than half of the comments during the discussions while least outspoken members make a very small portion of the comments.^[37]

3. Moderated/controlled settings of public meetings In order to minimize the potential effects of participants' demographic and cognitive characteristics on conversations, public meetings or consensus conferences tend to be carefully moderated and guided by facilitators. In such artificial setting, participants may behave in different ways that may differ from what is likely to occur in real-world discussions.^[38]

4. Spillovers from public meetings to real-world discussion

The social implication effect of follow-up media coverage of public meetings or other engaging events may help transfer issues from these small group discussions to the broader community. However, in the case of the U.S., a spillover effect from public meetings into media discourse are minimal at best.^[39]

5. Knowledge gap issues

Public meetings and consensus conferences may create knowledge gaps between high SES and less SES participants. The demographic, prepositional and cognitive differences between two groups in public meeting may lead to differing outcomes of public engagement. For example, highly educated participants may learn more from discussions and dominate the conversation while less educated members listen to their arguments. Furthermore, only small proportions of the population who may be already informed attend public meetings while the majority of the population who may need information the most do not. In such case, any public engagement effort may widen existing gaps further.

See also

For examples of public engagement, see also:

• Citizens' assembly
• Community x-change
• Internet 2.0
• Participatory democracy
• Participatory economics
• Participatory justice
• Public participation

References

1. Rowe, G.; Frewer, L. J. (2005). "A typology of public engagement mechanisms". Science, Technology, & Human Values. 30 (2): 251. CiteSeerX 10.1.1.546.7172. doi:10.1177/0162243904271724. S2CID 19433262.
2. Evans, J. H. (2020-10-02). "Can the Public Express Their Views or Say No Through Public Engagement?". Environmental Communication. 14 (7): 881–885. doi:10.1080/17524032.2020.1811459. ISSN 1752-4032. S2CID 222074307.
3. Tan, Kenneth Paul. "Public Engagement: The Gap between Rhetoric and Practice". Civil Service College of Singapore. Retrieved 15 January 2015.
4. Scheufele, Dietram A.; Krause, Nicole M.; Freiling, Isabelle; Brossard, Dominique (2021-04-30). "What we know about effective public engagement on CRISPR and beyond". Proceedings of the National Academy of Sciences. 118 (22). Bibcode:2021PNAS..11804835S. doi:10.1073/pnas.2004835117. ISSN 0027-8424. PMC 8179128. PMID 34050014.
5. Guidry, Jeanine P.D.; Jin, Yan; Orr, Caroline A.; Messner, Marcus; Meganck, Shana (2017). "Ebola on Instagram and Twitter: How health organizations address the health crisis in their social media engagement". Public Relations Review. 43 (3): 477–486. doi:10.1016/j.pubrev.2017.04.009.
6. Jeanine P.D., Guidry; Jin, Yan; Orr, Caroline A.; Messner, Marcus; Meganck, Shana (2017). "Ebola on Instagram and Twitter: How health organizations address the health crisis in their social media engagement". Public Relations Review. 43 (3): 477–486. doi:10.1016/j.pubrev.2017.04.009.
7. Bloomfield, D., Collins, K., Fry, C. and Munton, R., 2001: Deliberation and inclusion: vehicles for increasing trust in UK public governance? Environment and Planning C: Government and Policy 19, 501-513
8. Measham, T.G. Brake; Robinson, C.J; Larson, S. Richards; Smith, T. (2011). "NRM engagement between remote arid communities and government agencies: Success factors from Australia". Journal of Arid Environments. 75 (10): 968–973. doi:10.1016/j.jaridenv.2011.04.018.
9. There are a number of organizations, like California's Institute for Local Government, that provide public officials with analytic constructs to decide which approaches to use and evaluate the results. See, www.ca-ilg.org/publicengagement.
10. ScienceBlogs. (n.d.). ScienceBlogs. Retrieved November 13, 2011, from http://scienceblogs.com/
11. nsf.gov - S&E Indicators 2010 - Chapter 7. Science and Technology: Public Attitudes and Understanding - Public Knowledge About S&T - US National Science Foundation (NSF). (n.d.). nsf.gov - National Science Foundation - US National Science Foundation (NSF). Retrieved November 12, 2011, from "NSF.gov - S&E Indicators 2010 - Chapter 7. Science and Technology: Public Attitudes and Understanding - Public Knowledge About S&T - US National Science Foundation (NSF)". Archived from the original on 2011-10-01. Retrieved 2011-11-14.
12. Bhattacharjee, Y. (2010). "NSF board draws flak for dropping evolution from Indicators". Science. 328 (5975): 150–1. doi:10.1126/science.328.5975.150. PMID 20378779.
13. Corley, E. A.; Scheufele, D. A. (2010). "Outreach Going Wrong?". The Scientist. 24 (1): 22.
14. Calice, Mikhaila N.; Beets, Becca; Bao, Luye; Scheufele, Dietram A.; Freiling, Isabelle; Brossard, Dominique; Feinstein, Noah Weeth; Heisler, Laura; Tangen, Travis; Handelsman, Jo (2022-06-15). Baert, Stijn (ed.). "Public engagement: Faculty lived experiences and perspectives underscore barriers and a changing culture in academia". PLOS ONE. 17 (6): e0269949. doi:10.1371/journal.pone.0269949. ISSN 1932-6203. PMC 9200360. PMID 35704652.
15. Dudo, Anthony; Besley, John C. (2016-02-25). "Scientists' Prioritization of Communication Objectives for Public Engagement". PLOS ONE. 11 (2): e0148867. doi:10.1371/journal.pone.0148867. ISSN 1932-6203. PMC 4767388. PMID 26913869.
16. Scheufele, DA; Turney, J (2006). "Messages and heuristics: How audiences form attitudes about emerging technologies". Engaging Science: Thoughts, Deeds, Analysis and Action.
17. Drummond, Caitlin; Fischhoff, Baruch (2017-08-21). "Individuals with greater science literacy and education have more polarized beliefs on controversial science topics". Proceedings of the National Academy of Sciences. 114 (36): 9587–9592. doi:10.1073/pnas.1704882114. ISSN 0027-8424. PMC 5594657. PMID 28827344.
18. Salmon, Charles T.; Cho, Hyunyi (2007-06-01). "Unintended Effects of Health Communication Campaigns". Journal of Communication. 57 (2): 293–317. doi:10.1111/j.1460-2466.2007.00344.x. ISSN 0021-9916.
19. Peter, Christina; Koch, Thomas (2016). "When Debunking Scientific Myths Fails (and When It Does Not): The Backfire Effect in the Context of Journalistic Coverage and Immediate Judgments as Prevention Strategy". Science Communication. 38 (1): 3–25. doi:10.1177/1075547015613523. ISSN 1075-5470. S2CID 53698648.
20. Wills, Matthew (2017-04-03). "The Backfire Effect". JSTOR Daily. Retrieved 2022-04-05.
21. Besley, John C.; Newman, Todd P.; Dudo, Anthony; Tiffany, Leigh Anne (2020-09-02). "Exploring scholars' public engagement goals in Canada and the United States". Public Understanding of Science. 29 (8): 855–867. doi:10.1177/0963662520950671. ISSN 0963-6625. PMID 32878551. S2CID 221477041.
22. Rose, Kathleen M.; Markowitz, Ezra M.; Brossard, Dominique (2020-01-07). "Scientists' incentives and attitudes toward public communication". Proceedings of the National Academy of Sciences. 117 (3): 1274–1276. doi:10.1073/pnas.1916740117. ISSN 0027-8424. PMC 6985784. PMID 31911470.
23. "More Now Say It's 'Stressful' to Discuss Politics With People They Disagree With". Pew Research Center - U.S. Politics & Policy. 2018-11-05. Retrieved 2022-04-05.
24. Scheufele, Dietram A. (2014-09-15). "Science communication as political communication". Proceedings of the National Academy of Sciences. 111 (supplement_4): 13585–13592. Bibcode:2014PNAS..111S3585S. doi:10.1073/pnas.1317516111. ISSN 0027-8424. PMC 4183176. PMID 25225389.
25. Select Committee on Science and Technology Third Report CHAPTER 5: ENGAGING THE PUBLIC . (2010, April 1). parliament.uk . Retrieved November 12, 2011, from https://publications.parliament.uk/pa/ld199900/ldselect/ldsctech/38/3807.htm#a39
26. Americans Discuss Social Security «  AmericaSpeaks. (n.d.). AmericaSpeaks. Retrieved November 13, 2011, from "Americans Discuss Social Security « AmericaSpeaks". Archived from the original on 2012-07-08. Retrieved 2011-11-14.
27. Listening to the City. (n.d.). Listening to the City. Retrieved November 12, 2011, from http://www.listeningtothecity.org
28. Voices & choices Report On The Public's Priorities For Northeast Ohio's Future. (n.d.). voiceschoices.org. "NSF.gov - S&E Indicators 2010 - Chapter 7. Science and Technology: Public Attitudes and Understanding - Public Knowledge About S&T - US National Science Foundation (NSF)". Archived from the original on 2011-10-01. Retrieved 2011-11-14.
29. Joss, Simon (1998). "Danish consensus conferences as a model of participatory technology assessment: An impact study of consensus conferences on Danish Parliament and Danish public debate". Science and Public Policy. 25: 2–22 – via Oxford Academic.
30. Duckett, Catherine (2021). "Nights at the museum: integrated arts and microbiology public engagement events enhance understanding of science whilst increasing community diversity and inclusion". Access Microbiology. 3 (5): 000231. doi:10.1099/acmi.0.000231. PMC 8209632. PMID 34151182.
31. Rawlinson, Katherine (2021). "Family-focused campus-based university event increases perceived knowledge, science capital and aspirations across a wide demographic". International Journal of Science Education, Part B. 11 (3): 273–291. doi:10.1080/21548455.2021.1971319.
32. Steven, R., Barnes, M., Garnett, ST., Garrard, G., O'Connor, J., Oliver JL., Robinson, C., Tulloch, A., Fuller, RA. (2019). "Aligning citizen science with best practice: Threatened species conservation in Australia". Conservation Science and Practice. 1 (10). doi:10.1111/csp2.100.{{cite journal}}: CS1 maint: multiple names: authors list (link)
33. Hand, E. (2010). "Citizen science: People power". Nature. 466 (7307): 685–687. doi:10.1038/466685a. PMID 20686547. S2CID 205057718.
34. C. Doyle; R. David; Y.Li; M. Luczak-Roesch; D. Anderson; C.M Pearson (30 June 2019). "Using the Web for Science in the Classroom: Online Citizen Science Participation in Teaching and Learning". Proceedings of the 10th ACM Conference on Web Science. pp. 71–80. doi:10.1145/3292522.3326022. ISBN 978-1-4503-6202-3. S2CID 195777103. Retrieved 30 July 2019.
35. Crain, Rhiannon; Cooper, Caren; Dickinson, Janis L. (2014). "Citizen Science: A Tool for Integrating Studies of Human and Natural Systems". Annual Review of Environment and Resources. 39: 641–665. doi:10.1146/annurev-environ-030713-154609. S2CID 154358290.
36. de Santis, V. S., and T. Renner. 1997. Democratic traditions in New England town meetings: Myths and realities. In Annual Convention of the Midwest Political Science Association. Chicago, IL.
37. Merkle, D. M. (1996). "The polls-Review-The National Issues Convention Deliberative Poll". Public Opinion Quarterly. 60 (4): 588–619. doi:10.1086/297775.
38. Scheufele, A. D. (2011). Modern Citizenship or Policy Dead End? Evaluating the need for public participation in science policy making, and why public meetings may not be the answer . Paper #R-34, Joan Shorenstein Center on the Press, Politics and Public Policy Research Paper Series. Harvard University. Cambridge, MA. Retrieved from http://www.hks.harvard.edu/presspol/publications/papers/research_papers/r34_scheufele.pdf
39. Dudo, A. D.; Dunwoody, S.; Scheufele, D. A. (2011). "The emergence of nano news: Tracking thematic trends and changes in U.S. newspaper coverage of nanotechnology". Journalism & Mass Communication Quarterly. 88 (1): 55–75. doi:10.1177/107769901108800104. S2CID 143289969.