Skip directly to content

Blog Feed

Ray Dearborn's picture

Upwell's Distributed Network Campaigning Method

on February 28, 2013 - 5:32pm

The mission of Upwell is to condition the climate for change in marine conservation, and to ready people to take action. In order to do this, our team sifts through the vast amount of real-time online content about the ocean, and amplifies the best of it. Upwell’s campaigning model capitalizes on the insights we glean from Big Listening and other curation efforts, and responds to the currents of online conversation. Through an iterative process of lots and lots of campaign testing, we find ways to create spikes of attention in conversations. Ultimately, we hope to raise the day-to-day, week-to-week, month-to-month, year-to-year baseline of those conversations.

What is an Upwell campaign?

Upwell’s campaigning model combines a few key elements. Our campaigns are attention campaigns, focused on raising attention to ocean issues. They are minimum viable campaigns, operating on short time-frames and focused on rapid delivery of content, continuous learning and iteration. They are run and amplified across a distributed network, rather than being housed on, and amplified by way of our own platforms. 

The Attention Campaign

The nonprofit community has deeply-held ideas of what constitutes a campaign. Often, organizations build campaigns with institutional goals (e.g.: awareness, list-building, advocacy and fundraising campaigns), and compete with other entities in the same sector/issue space. Upwell’s attention campaigns operate on a different plane, one in which success (greater attention) elevates the work of everyone in Team Ocean, and is tied to no particular institutional outcome other than generating conversation. 

What we do with attention campaigns is try to drive more attention to existing content and actions that are not on our properties. They’re not associated with our brand. We use this loosely held connection, tying into the momentum of the news cycle, and being strategically opportunistic in the pursuit of creating spikes in attention. 

We focus on shareability, and measure our success by the same, simple attention metric we use to measure online conversations: social mentions. Social mentions are the currency of attention, and represent small bits of action. In contrast, awareness is a less meaningful measurement, representing what someone thinks they might do, not what they have done. 

Over time, we believe that increased attention to ocean issues will raise the daily baseline of conversation about ocean issues. We have been experimenting with trying to understand what makes baselines go above the expected, or historical level (i.e., what causes spikes in conversation), with an eye toward making these increases in attention sustainable.

The Minimum Viable Campaign

“You can't just ask customers what they want and then try to give that to them. By the time you get it built, they'll want something new.” - Steve Jobs

On the advice of Sean Power, Data Scientist at Cheezburger, Upwell has adapted an agile development principle from the  lean startup movement - the minimum viable product. Our campaign lifecycle embodies the Build-Measure-Learn cycle that software developers have used in order to quickly release products with the minimum amount of functional features, in order to gather immediate insight that can inform later iterations. 

The cycle of agile software development

Through our minimum viable campaigns, we employ ongoing, iterative, continuous delivery of content, resisting our urges toward perfection and providing irreverent, timely, contextual content to audiences immediately instead of strategizing for six months, or a year. We focus on the quickest, dirtiest thing we can get out the door that we think will have a measurable effect on a conversation. 

Our campaigns have short lifecycles - anywhere from a couple hours to a few days, and they are inspired and informed by hot news that feels really immediate to those campaigns. We move very rapidly through a process of hatching an idea, finding or creating the campaign product(s), putting it out into the world and getting back data. We are constantly learning how to be more effective. In our first year running over 160 minimum viable attention campaigns, we have learned that even a tiny bit of effort can make a huge difference in how campaigns get picked up.

Combining These Models

The minimum viable campaign model could be applied to not just attention campaigns, but also fundraising, advocacy, or other types of campaigns. Likewise, an attention campaign could certainly be run at different time scales. For instance, there is no reason why Red Cross couldn’t start doing minimum viable campaigns. Keeping everything else the same, they could tighten up their campaign time cycles and run experimental campaigns to engage their base in different ways. Red Cross could also start running attention campaigns. If they believed that Ushahidi was doing really good work, they could run an attention campaign, pointing at Ushahidi’s work and amplifying attention to it. This could turn out to be a faster path to achieving their own mission.

By applying both these models, Upwell has crafted a new way of campaigning that is easily delivered, measured, and adapts to the ever-changing sea of conversation. In summary, through our campaigns, Upwell:

  • Surfs existing conversations in order to increase and expand attention.
  • Measures social mentions (rather than policy outcomes, petition signatures, or public opinion) to evaluate the success of our campaign efforts.
  • Delivers, measures, and learns from campaigns on a short time cycle, embedding lessons and insights immediately. We sacrifice perfection.
  • Collaborates with a network of ocean stakeholders and curates a diverse set of existing ocean content, rather than building on our own brand, and creating our own content. Our campaigns are not aligned with Upwell program priorities or policy goals, but instead amplify attention to the priorities and goals of those in our network.
  • Runs our campaigns across a distributed network of ocean communicators, rather than relying on our own platforms as information hubs.

The Upwell Network

The key to our campaigns’ success is in our network. Our attention campaigns are amplified not by us, or by a dedicated base of supporters we’ve built over the years, but rather by the network of ocean communicators that we regularly contact through the Tide Report, our social media channels, and our blog. We call this “running a campaign across a distributed network.” It’s more of a syndication model than a direct-to-consumer model.

We built our network proactively to respond to several trends. With the rising cacophony of the internet, the rapidly increasing pace at which news spreads, and the shift toward people finding news through their friends on social media channels rather than getting it directly from “official” channels, we decided to approach network campaigns in a new light. It would have been cost prohibitive to buy the attention (through ads or purchasing email lists) or to build a world-class, unbranded media hub. Rather than collect a large set of official MOU’s and partner logos to put up on our website, we built a loosely held, distributed network. We’ve reached out to nodes of people who control the communications channels that reach lots of people who are interested in ocean issues. We’ve been scrappy and ruthless about who we put into that distributed network, trying to make it diverse and ensure the reach is big.

Campaigning across a distributed network means that we have that golden ticket of communications - message redundancy - but those redundant messages are all tailored by the individual nodes in our network for their audiences. It’s the job of the individual people in our network to know their audience really well. They take our messages and content and they translate them out to their audiences through the communications channels they maintain. 

As a point of comparison, Upworthy, a similar effort that launched just after Upwell and that shares our goal of making social change content more shareable and “viral,” approached the problem of distribution from a different angle. Rather than build their own network through which they could distribute the content they curate, they built their own media hub, repackaging content under the Upworthy banner, and rapidly scaling up an audience and brand of their own. This model certainly brings eyes to worthy content, but doesn’t (yet) effectively pass on engagement to the organizations and individuals it supports - it retains that engagement for its own channels.

We wanted to build an issue-specific network, and through our networked campaigns, strengthen our network’s members’ and supporters’ potential for future action. 

Below are the values that guide Upwell in building and strengthening our distributed network:

  • Trust: we share only science-based content, ensuring that other science-based institutions know that the content we share is trustworthy.
  • Transparency: we share our campaign and big listening data with our network, so that they can apply our lessons in their own work.
  • Brand-agnostic: we work as willingly with Greenpeace as we do with Deep Sea News, as we do with the Facebook page “I Fucking Love Science.” We will share an organization or individual’s content or campaign, as long as it promotes ocean conservation goals and fits our curation criteria. Often, promoting content from an array of brands meant releasing control of the message.
  • Issue-agnostic: We aren’t working just on overfishing, or GMO salmon, or catch shares to cultivate the network. We amplify any ocean campaign or content as long as it fits our curation criteria.
  • Personal: We build relationships with humans, not organizations. The liveliest online conversations happen between people, not institutions. We model the authentic behavior of the internet.
  • Generous: We provide small bits of advice and feedback to help our network do better. If their work will get more people talking about the ocean online, it fits with our mission.

***

We've spent the last six weeks reflecting on our pilot project, and want to share our results with you. This post is one in a series of pieces about what we've learned over the last 10 months.

If you like this post check out:

Matt Fitzgerald's picture

Upwell's Spike Quantification of the Ocean Conversation

on February 28, 2013 - 4:10pm

What is a Spike?

A spike is a significant increase in online attention for a particular topic. When you graph those social mentions, you can actually see that burst of attention ‘spike’ the graph -- hence the name. 

We have been observing spikes in the wild, so to speak, since the beginning of Upwell. It’s a concept that is at least somewhat familiar to anyone who has ever described a video as “viral,” or checked out the list of the most shared articles on The New York Times' website. A lot of people, sharing one thing, over a short time, creates a spike. In the world of Big Listening, that one thing they share can actually be a large number of different things on the same topic, but the general point remains the same. Surges in attention create spikes. So how do you measure one?

Let’s look at a graph of the sustainable seafood conversation from Summer 2012:

Social mentions for Upwell’s Sustainable Seafood keyword group vs. Upwell’s Sustainable Seafood Baseline, June 1, 2012 - August 1, 2012.

It seems pretty clear that there are two spikes in this time period. One appears on June 8, the other on June 16. But what about the other days? How far above the Baseline does social mention volume have to be in order to qualify as a spike? We set out to find a way to compare spikes that would answer the question.

Before we dive in, it’s important to note that social mention volume for a given day is a construct. We decided to use a day as the operating unit of time both because the tools we have available to us use that temporal distinction, and because a day as a unit of measurement is widely understood. That is not to say that one couldn’t decide to measure spikes by the hour, by the minute, or by some other amount of time. We made a conscious decision to build our initial definition of a spike around the day, but infinite other options exist as well. 

A second caveat is that focusing on spikes may obscure what is actually making up the long tail of post volume. Upwell talks about, and quantifies, much of this activity as the Baseline, but there may be other small-to-medium bursts of attention that last more than a day and consequently don’t visually ‘spike’ a graph in the same way (think of a multi-day increase in attention as a hump or a mesa, rather than the taller, more angular spike). Spikes look good on charts, and they help push conversations into the wider internet, but they are not the whole story of an online topic. We long for a day when tools for Big Listening allow us to view topic volume graphs like geologists look at cross-sections of rocks -- that day is not here yet. 

With those caveats out of the way, we can return to our earlier question: what is a spike? Remember from our discussion of Baseline quantification that Upwell’s analysis is designed to inform a set of interventionist activities. We:

  • identify and target high-value items to campaign on;
  • compare the relative size of different ocean sub issues (e.g. sharks vs. whales); and
  • measure the impact of our campaigns.

Spike quantification informs our campaigning and provides one measure of results. We’re not interested in just contributing to the noise around a given ocean topic, we actually want to help a signal to emerge. Spikes are those signals. Evaluating opportunities to campaign becomes a much more concrete activity when you know exactly how many social mentions are needed to break through the regular volume of conversation.

After examining historical social mention volume for our Sustainable Seafood and Overfishing keyword groups, we calculated a variety of statistical thresholds for the exported data and compared the results to our measured campaign and spike data. As discussed earlier, Upwell’s Baseline calculations are derived from the insight that our primary ocean topics each demonstrate a weekly periodicity. Similarly, in calculating potential thresholds for what constitutes a spike, we started with that same insight and then calculated various multiples of standard deviation above the average (mean) value for that day of the week. Because standard deviation measures how spread out the values within a data set are, using it to measure a particular value’s variation from the “normal” value of that data set is a good way to test for a spike. Spikes are visible because they’re outliers, and that’s what the standard deviation threshold(s) tests. 

Day-of-the-week values for the Sustainable Seafood Baseline, along with the Sustainable Seafood mean, and mean +1x, +1.5x and +2x standard deviations (10/17/11 - 1/29/13).

As seen above, the standard deviation thresholds are higher than both the Baseline and the mean. Graphing those thresholds against our campaign and event records revealed that the one standard deviation threshold was the most accurate representation of what we were observing on a day-to-day basis.

Social mentions for Upwell’s Sustainable Seafood keyword group vs. Upwell’s Sustainable Seafood Baseline vs. ‘Mean + 1 Standard Deviation’ Spike Threshold (June 1, 2012 - August 1, 2012)

Upwell defines a spike as occurring when the social mention volume for a given day meets or exceeds one standard deviation from the mean of all recorded values for that same day of the week.

While a critic might accuse us working backwards to find the threshold that gives the best fit, we would actually agree. Sustainable Seafood and Overfishing are the topics that we know the best -- because we’ve monitored them and campaigned on them with the most focus -- and we were looking for a metric that would have practical implications for attention campaigns. As mentioned before, we remain open to other spike quantification approaches, but this one is our preferred option, given what we know right now.

What Does Spike Quantification Tell Us?

Upwell’s spike quantification methodology is in alpha, so to speak, and going forward we will look to improve it. The possibilities for more comparative measures of success are numerous. One thing is certain, however: applying a spike quantification lens to our work is illuminating.

Spike comparison beta methodology?

The following graphs show our first Winter in 2011 and most recent Winter in 2012 working in the Overfishing and Sustainable Seafood conversations. Both one standard deviation and two standard deviation threshold lines are included for reference.

The comparison in time periods for both conversations is dramatic. There is a noticeable increase in spike frequency (the number of spikes), spike volume or  “spikiness” (see: the number of spikes exceeding two standard deviations), and in the overall volume of conversation in the time period as measured against the Baseline. To be blunt: this is what success looks like.

Sustainable Seafood: Winter 2011

Social mentions by day for Upwell’s Sustainable Seafood keyword group, as compared to the Sustainable Seafood Baseline, as well as to spike thresholds of one standard deviation and two standard deviations above the day-of-the-week mean (10/17/2011 - 1/31/12). Total post volume: 45,255 social mentions over 107 days. Average volume / day: 423 social mentions.

Sustainable Seafood: Winter 2012

Social mentions by day for Upwell’s Sustainable Seafood keyword group, as compared to the Sustainable Seafood Baseline, as well as to spike thresholds of one standard deviation and two standard deviations above the day-of-the-week mean (10/1/2012 - 1/29/2013). Total post volume: 66,456 social mentions over 121 days. Average volume / day: 549 social mentions.

Overfishing: Winter 2011

Social mentions by day for Upwell’s Overfishing keyword group, as compared to the Overfishing Baseline, as well as to spike threshholds of one standard deviation and two standard deviations above the day-of-the-week mean (10/17/2011 - 1/31/12). Total post volume: 211,799 social mentions over 107 days. Average volume / day: 1,979 social mentions.

Overfishing: Winter 2012

 

Social mentions by day for Upwell’s Overfishing keyword group, as compared to the Overfishing baseline, as well as spike threshholds of one standard deviation and two standard deviations above the day-of-the-week mean (10/1/2012 - 1/29/13). Total post volume: 409,692 social mentions over 121 days. Average volume / day: 3,386 social mentions.

***

We've spent the last six weeks reflecting on our pilot project, and want to share our results with you. This post is one in a series of pieces about what we've learned over the last 10 months.

If you like this post check out:

Upwell's Ocean Conservation Baseline Methodology

Upwell's Distributed Network Campaigning Method

Conversation Metrics for Overfishing and Sustainable Seafood

Matt Fitzgerald's picture

Upwell's Ocean Conversation Baseline Methodology

on February 28, 2013 - 2:49pm

What is a Baseline?

When Upwell says that we listen to the ocean conversation, what that means in practice is that we're conducting Big Listening on English-language conversations in the following eight "ocean conversation" topic areas: Overfishing, Sustainable Seafood, Marine Protected Areas (MPAs), Oceans (broadly defined), Cetaceans (whales and dolphins), Sharks, Tuna, Gulf of Mexico and Ocean Acidification. For each topic, both real-time and historical data provide essential context for understanding the volume, evolution and characteristics of the overall conversation.

Each topic we monitor is characterized and defined by a set of search terms (including exclusions) that we refine on an ongoing basis. While we recognize the limitations of “keyword groups,” such as their reliance on text-based results and the absence of  contextual awareness, they do provide a powerful tool for analyzing online attention. The development and active refinement of keyword groups is at the heart of Big Listening methodology.

We use Big Listening in order to:

  • identify and target high-value items for campaign purposes,
  • compare the relative size of different ocean sub issues (e.g. sharks vs. whales), and
  • measure the impact of our campaigns.

Since Upwell is a campaign agency (among other things), we needed a way to characterize these conversations as they exist, absent our interventions.

Enter the Baseline.

Baselines help us to anchor campaign performance targets in measures of past conversational volume. We set goals informed by the Baseline (as well as by spikes), and then campaign to meet and exceed those targets.

Upwell informally defines a conversation’s Baseline at the point below which the daily volume doesn’t drop. It can be thought of as a floor (although it is often quite high - in the tens of thousands for a conversation like Cetaceans), or as the number of social mentions performed each day by the topic’s diehard conversationalists. If everyone else left the party, the Baseline would still be there, dancing by itself.

Robyn - euro popstar, solitary dancer, human Baseline metaphor

The Baseline: Up Close and Personal

Upwell’s Baseline methodology has evolved to capture the highly dynamic conversations we watch, especially cyclical variations by day of week. These cyclical variations often result from usage and posting patterns. For example, people tend to talk on the internet when they’re at work. Over the course of our pilot grant, Upwell has used three different version of baseline methodology to better measure the dynamic online conversation space:

  • Baseline v1.0: The lowest level of daily social mentions for a given conversation, for a given period (implemented using Upwell topical keyword groups) [in use through late August 2012]
  • Baseline v2.0: The median daily social mentions for a given conversation/keyword group for a given period [in use through mid-November 2012]
  • Baseline v3.0: The average of the lowest 10% of social mention values for a topic on a given day of the week [in use through early January 2013]
  • Baseline v3.1: The average of the lowest 20% of social mention values for a topic on a given day of the week [currently in use]

Our Baseline quantification methodology was created with input from leaders in the field including: K. D. Payne, Chairman & founder of Salience/ KDPaine & Partners, and co-author of the recently released Measuring The Networked Nonprofit; leading nonprofit technologist and Packard Fellow, Beth Kanter; and a senior Educational Policy Analyst for a leading national measurement/ social statistics firm contracted by the Gates Foundation, the Broad Foundation and the U.S. Department of Education.

To calculate the Baseline for a particular topic we begin by compiling all available social mention data for the period since we started monitoring it (mid-October 2011 or later, depending on the topic). We then disaggregate the data by the day of the week in order to deal with cyclical variations in post volume and compare Mondays to Mondays, and Sundays to Sundays. Once that’s done we calculate the average (mean) of the lowest 20% of values for each of the seven days. Taken together, those day-of-the-week values are what we refer to as the Baseline.

We selected the average (mean) to establish a specific value for each day of the week for three reasons. First, the mean is the starting point for calculating standard deviations used in our spike quantification methodology. Second, given the small size of most ocean conversations, the mean is the most typically consistent and available measurement when analyzing the conversation on a by day-of-the-week basis.

Baseline Social Mentions by Day-of-Week for Upwell’s Sustainable Seafood keyword group (10/17/11 - 1/29/13)

These daily Baseline values are then graphed against social mention data over time. The graph below shows the result.

Social mentions for Upwell’s Sustainable Seafood keyword group vs. Upwell’s Sustainable Seafood Baseline, June 1, 2012 - August 1, 2012

Context and Challenges for Baseline Quantification

As seen in the overfishing and sustainable seafood conversations, day-of-week periodicity is highly evident in social mention volume, largely driven by mention increases during U.S. working hours across Eastern to Pacific time zones, and with substantial drop offs on weekends. Using the current Baseline as a reference for setting campaign goals removes the disincentives present in previous Baseline quantifications to campaign on lower volume days, and gives a more accurate picture of success on all campaign days.

Our campaign efforts, along with unexpected conversational developments (think: the release of a new report or the arrival of a natural disaster) require us to add new terms to the relevant keyword group so that our work, or online mentions of some relevant, unanticipated development, is captured in our search terms. In the inevitable cases where we find noise in the results returned by particular terms (new and old), we respond by tightening or removing those terms from the keyword group.

Since keyword groups have keywords added and subtracted on an ongoing basis, there are some inherent challenges. How should Baseline calculations account for these changes? How and when should Baselines be refreshed? Should campaign targets be refreshed retrospectively? How should we treat a spike that loses or gains volume with a refreshed keyword group? Moving forward, Upwell will continue to drive innovations in our baseline methodology and in the integration of that methodology into our campaign process.

Finding the Baseline

Developing a Baseline for a topic or conversation requires an iterative process of definition, testing and measurement. And once you’ve got it, you have to refresh it to account for conversational evolutions. It’s like a marriage in that way, or so we’re told.

Measuring something requires you to define it. The trick in measuring a conversation is that conversations change over time as participants engage in dialogue. As conversations dynamically evolve over time, so too do the methods of expression (i.e. terminology, imagery, metaphors, etc.), the composition of participants, and the accompanying platforms. Much like species evolution, these changes are not always in a direction that we perceive to be fruitful. Evolution can lead to progress just as it can lead to dead-ends or fragmentation. Because of this, no conversational “listening” can ever be exhaustive (as some elements of the conversation will always be overlooked), nor can it be perfectly accurate (as noise will always creep in). This is why we continue to add and subtract terms to our Baseline keyword groups. Ongoing monitoring and modification of a Baseline keyword group is the most effective way to keep that keyword group refreshed and accurate. Since designing a keyword is as much about selective addition as selective omission, our ability to effectively evolve keywords and keyword groups is informed just as much by our listening practice with Radian6 as it is by our personal listening, professional network and subject matter expertise. No robot can do this, although some are trying, and interns probably can’t do it either.

In “baselining” a conversation, Upwell begins by developing a conceptual framework for the topic in question. For the purposes of explanation, let’s imagine our topic is Marine Debris. We would begin our Baseline development process by outlining the conceptual and temporal boundaries of analysis for marine debris. The temporal aspect is important because a keyword group developed for one time period may lose significant accuracy (and utility) when applied to another period. For the conceptual outlines we often make use of a mind map such as the one for Oceans shown partially below.

Upwell mind map for Oceans keywords, January 2012

For marine debris, the concept map would include items such as marine trash, the pacific gyre, marine plastics, great pacific garbage patch, seaplex (minus exclusions for the botanical shampoo of the same name), albatross AND plastic. The concept map would also include people, campaigns, expeditions and organizations such as Miriam Goldstein (a marine debris expert), The Trash Free Seas Alliance, the Plastiki, and Seaplex.

The concept map becomes a design artifact for further conversations about the conversation. Although we sometimes shortcut this process in the interests of time, we refine the map through a series of discussions and email exchanges with subject matter experts and knowledgeable people in the industry or industries at play.

Once we have a solid map of the conversation, we turn the map into a series of keywords. Keywords are textual search terms, much like something you might google. A keyword for Upwell, for example, would be “Upwell.” But it might also be a distinctive phrase (or fragment thereof) such as our tagline, “the ocean is our client.” A keyword, in this way, can actually be a number of elements (such as multiple words in a phrase) despite its singular form. This fact will become more important as we discuss more of those elements. A collection of keywords is called a keyword group, and search results for keyword groups are the foundational output of Big Listening.

A couple wrinkles make the construction of keywords and keyword groups significantly more challenging that one might expect. The first is noise. When you type something into a Google search bar and click on a result, you are deploying a potent combination of Google’s massive computational power, billions of dollars in said company’s algorithmic investments, and the concentrated smarts of your own interpretive brainpower. The last element is particularly important. Whereas Google displays dozens, if not thousands of search results for you to choose from -- and then asks you to filter those results-- the keyword queries we construct for Big Listening must be built so as to filter out as much noise as possible. To return to friend-of-Upwell Miriam Goldstein, mentioned above for her marine debris expertise, a well-constructed keyword group for that subject would probably not include her name as a standalone keyword -- the reason being that she talks about other, non-marine-debris subjects as well. Entering “Miriam Goldstein” as a keyword nets you any mention of her full name, whether that takes the shape of a blog post about ocean trash or a friend’s tweet referencing her attendance at a particularly yummy brunch meeting. What you leave out of a keyword or keyword group is as important as what you put in.

Pruning out extraneous results through proper keyword construction brings us to the second wrinkle: exclusions. Exclusions are also textual search terms, but their purpose is to filter out results that match their terms. Exclusions can be tied to specific keywords or to entire keyword groups. A well-constructed (or scoped) exclusion can be the difference between finding online mentions of sharks, the creatures, or finding online mentions of the San Jose Sharks, the hockey team, creatures of a different sort. Exclusions can filter out things beyond keywords (such as entire categories of website domains, particular geographies of origin, or what a computer determines to be a particular language) using a variety of tools used in Big Listening-- source filters, to give one example.

Another wrinkle in keyword construction is proximity. Proximity is not available in every tool that might be applied in a Big Listening process, but it is present in Radian6, our tool of choice at the moment. Proximity is a modifier that can be applied to two or more words in a keyword, say “marine” and “debris.” Proximity, denoted by “~”, tells the tool/service how close a set of words must be in order to return a match. Closeness basically means: how many other words come in between? If we were to set proximity to zero for “marine debris,” Radian6 would return only items that include that exact phrase. If we set proximity to three, for example, we might get results such as “marine layer clotted with debris.” That distinction becomes increasingly important when Radian6 scrapes long forum discussions, news articles, or blog posts in which a topic might be mentioned in an extremely peripheral manner. Proximity provides another tool to scope a given keyword and focus the results in a particular way.

Keyword development feeds into an ongoing measurement process of Scope / Test / Adapt / Share. The cycle is presented below.

SCOPE

Initial investigation

1. Outline the conceptual and temporal boundaries of analysis for the topic

2. In consultation with subject-matter experts and other stakeholders, create a seed list of topics, subtopics, potential online influencers, and known online sources, events, and campaigns

TEST

Generating preliminary keywords

3. Use the seed list to develop initial keyword inputs for online search and social media monitoring services

4. Develop a more detailed set of keywords

5. Verify keyword accuracy and relevance using Radian6 to graph and spot-check search results, adding exclusion terms to filter extraneous results/noise, or various degrees of proximity to widen the net

ADAPT

Refining keyword groups

6. Share keyword lists with key informants (subject experts, foundation staff, campaigners etc.) and incorporate feedback (e.g., additional terms, scope adjustments)

7. Repeat steps 3, 4 and 5 with updated keywords

Cultivating and maintaining keywords

8. Campaign or otherwise monitor keyword group results on an ongoing basis

9. Update Baseline keyword groups with new inclusions and exclusions based on current events, campaigns and other developments (while always testing for the introduction of noise, per step 5)

10. On an as-needed basis, generally after at least three months of listening, share Baseline keyword groups with subject matter experts and other groups to gather feedback and potential improvements

SHARE

Exporting and preparing data

11. On a monthly, quarterly or to-order basis, export Big Listening data based on the most current keyword groups

12. Recalculate Baseline values

13. Graph and annotate charts with spike identifications

Packaging and distributing insights

14. Create reports, blog posts and other types of synthesis for external audiences

Improving the methodology

15. Gather feedback and process what we’ve discovered

16. Iterate our overall set of procedures

A crucial detail of the final stage of this process is the fact that exporting the data freezes it in time. To offer a contemporary example, all of the data in this report was current as of the end of January 2013, and then it was frozen in a spreadsheet. It’s important to remember that the conversations we monitor continue to change, even as we’re measuring and reporting on them. 

Because the exported data is a snapshot of results for a particular conversation’s keyword group, as it existed at a particular time (of export), from a particular tool (or combination of tools), the resulting values cannot and should not be separated from the keyword group that produced them. Furthermore, due to the item volume returned by some of some of the larger keyword groups, exporting data will sometimes produce variations in measurements for the same hour, day or time period. This variation is due to the tools we use and is generally extremely small given the scale of the topics we’re monitoring. These two factors combine to reinforce our belief that Big Listening data can only be fully interpreted if the underlying keywords are available -- anything less is a black box

***

We've spent the last six weeks reflecting on our pilot project, and want to share our results with you. This post is one in a series of pieces about what we've learned over the last 10 months.

If you like this post check out:

Upwell's Spike Quantification of the Ocean Conversation

Upwell's Distributed Network Campaigning Method

Conversation Metrics for Overfishing and Sustainable Seafood

Ray Dearborn's picture

An Upwell uprising? More from the Mobilisation Lab!

on February 12, 2013 - 11:36am

Greenpeace's Mobilisation Lab exists to provide Greenpeace and its allies:

a dynamic, forward-looking space to envision, test, and roll out creative new means of inspiring larger networks of leaders and people around the world to break through and win on threats to people and the planet.

Pretty rad, huh? The MobLab's focus on people-powered change, networked campaigning and digital innovation makes them one of our best peers in this space. We read each of their Dispatches voraciously, hungry for the insights from the case studies they provide.

Which is why we were thrilled when they posted on their blog today about our work. What most excites me are the testimonials from Greenpeace campaigners around the world, who will be testing our methods in their campaigns. Alex Rozenfarb from Greenpeace Israel noted:

Every day I am going to have a meeting and have something done up in a creative way. I’ve already bought this monitoring software and starting to make a standardization of how we do it in on a daily basis, I’m going totally in on her strategy.

We hope to hear back from Alex and other participants of Greenpeace's Digital Mobilisation Skillshare to understand what kind of lessons they learn in this new world of minimum viable campaigning.

Read more here: An Upwell uprising? GPDMS session on daily campaigning demonstrates potential for longer, deeper conversations on your issues

Ray Dearborn's picture

Spreading the good word of Big Listening and Minimum Viable Campaigning

on February 6, 2013 - 10:53am

This week, our illustrious leader, Rachel Weidinger, is present at the Greenpeace Digital Mobilisation Skillshare outside Barcelona. (Follow along on Twitter at #GPDMS). 

This morning, Rachel led a session with global campaigners and organizers on Upwell's model of Big Listening and campaigning (for more on Big Listening, check out Rachel's guest post on Beth Kanter's blog). She not only explained what we do - she also had participants break out into groups to apply Upwell's model to other issue areas. Believe it or not, what we do doesn't just work for the ocean conversation!

Brian Fitzgerald, Greenpeace's Head of Digital Networking and Mobilization, blogged his notes from the session, as well as his awesome work at using our methodology for Indonesia's rainforests. Check out the awesomeness he and his team made:

I thought Brian's post was so awesome, I left a comment for him. You should too! 

 

Pages