5  Transforming Data

Where the previous chapter dealt with selecting columns, filtering rows, and rearranging the resulting data set, this chapter will look into transforming data by calculating new values and summarizing groups of rows.

Chapter objectives

At the end of the chapter you will be able to:

• Compute new columns and changing existing values with mutate (Section 5.1)
• Recode values into new categories (Section 5.2)
• Group and summarize data (Section 5.3)
• Use pipelines (|>) to combine multiple analysis steps (Section 5.3.2)
• Check and convert column types, and format numbers (Section 5.4)
• Work with dates and times: extract parts, group, parse, format, and deal with time zones (Section 5.5)

As always, the chapter ends with a multiple choice quiz organized per section that you can use to test your knowledge and understanding. You can also expand the table below to see which functions are covered in this chapter.

NoteFunctions introduced in this chapter

Function	What it does	Section
mutate()	compute a new column or change an existing one	Section 5.1
round(), floor(), abs()	round, round down, or take the absolute value of numbers	Section 5.1
if_else()	recode into two options based on a condition	Section 5.2
recode_values()	recode values using a lookup of old and new values	Section 5.2
replace_values()	like recode_values, but leaves unmentioned values unchanged	Section 5.2
case_when()	recode using arbitrary conditions	Section 5.2
group_by()	group the rows of a data frame by one or more columns	Section 5.3
summarize()	compute summary statistics, one row per group	Section 5.3
n(), tally(), count()	the number of rows in each group	Section 5.3.3
ungroup()	drop all grouping levels	Section 5.3.4
summarize(.by=)	summarize with an implicit grouping	Section 5.3.4
mean(), median()	the mean or median of a column	Section 5.3
|>	pipe the result of one function into the next	Section 5.3
slice_max(), slice_min()	the row(s) with the highest or lowest value in each group	Section 5.3.7
slice_head(), slice_tail(), slice_sample()	the first, last, or a random sample of rows in each group	Section 5.3.7
as.numeric(), as.character()	convert a column to numbers or text	Section 5.4
format()	format numbers or dates as text	Section 5.4, Section 5.5
number()	format numbers with a fixed precision, prefix, or suffix (from scales)	Section 5.4
wday(), year(), month(), hour()	extract a part of a date or timestamp	Section 5.5
tally()	count the number of rows (per group)	Section 5.5
class()	the type of an object or column	Section 5.5
floor_date()	round dates down to e.g. the day, week, or month	Section 5.5
as.Date()	convert text or a timestamp to a date	Section 5.5
dmy(), mdy_hm(), …	parse text into a date or timestamp, given the order of the fields	Section 5.5
parse_date()	parse dates using a format string and locale	Section 5.5
tz()	the time zone of a timestamp	Section 5.5
with_tz()	convert a timestamp to a different time zone	Section 5.5
force_tz()	set the time zone of a timestamp, keeping the clock time	Section 5.5

Data used in this chapter: The Bechdel test

As a running example through this chapter, let’s have a look at the bechdel data, which contains information on the famous Bechdel test for female representation in films: does the film contain any scene where two women talk about something other than a man? This might seem a very low bar, but that will turn out to be a fairly naive assumption.

Note that this dataset is a simplified version of the data published by the 538 data-driven politics blog, now sadly discontinued. Although ABC (the new owner of 538) removed the earlier articles (literally the day that we wrote the first version of this chapter!), you can still read the original article using the archive.ph link: The Dollar-And-Cents Case Against Hollywood’s Exclusion of Women.

To download and read the data, we use the built-in download.file function and the tidyverse read_csv function:

Looking at the column names and types, can you take a guess at what each column means?

Suppose we want to know whether movies with better female representation give a better return on investment. What steps would we need to take to find this answer? And what if we want to know whether female representation is improving?

Thinking ahead for the next chapter on visualization: - what kind of plots did they use in the linked 538 article? - What data steps would be needed to produce the numbers for these figures? - Can you think of other useful visualizations for this data?

NoteColumns in the bechdel data - click to expand

Variable	Type	Explanation
imdb	character	IMDb identifier for the film
year	numeric (dbl)	Year of release
title	character	Title of the film
test	character	Detailed Bechdel-test outcome: one of ok, dubious, men, notalk, nowomen, sometimes with a -disagree suffix when the rating was contested
budget	numeric (dbl)	Production budget in US dollars (inflation-adjusted)
domestic_gross	numeric (dbl)	Domestic (US) box-office gross in US dollars (inflation-adjusted)
foreign_gross	numeric (dbl)	International box-office gross in US dollars (inflation-adjusted)

5.1 Changing or computing values with mutate

The essential tool in the data transformation toolbox is the mutate function. With mutate, you can create new columns or change the values in existing columns.

For example, if we want to know whether films that pass the test have a better return on investment, we need to compute the return on investment (dividing earnings by budget) and recode the 10 different values for test into a single yes or no (dummy / boolean) column.

The mutate() function

The mutate function makes it easy to create new variables or to modify existing ones.

For example, this computes the total gross earnings (international plus domestic) for each movie:

(You’ll notice R can display large numbers in scientific notation, i.e. using 1.3e7 to mean \(1.3 \times 10^7\), or 13 million. )

The syntax of mutate is similar to that of the filter() and select() functions introduced in the previous chapter: The first argument is the data frame to mutate, and then any number of additional arguments can be given to perform mutations. The mutations themselves are named arguments in the form column_name = calculation. The result of the calculation will then be stored in the named column, which can either be a new column or you can overwrite an existing column.

For calculations, you can use mathematical symbols such as +, / (division), and * (multiplication). There are also many useful functions that you can use in calculations, such as round() to round values, floor() to round values down to the nearest whole number, or abs() to take the absolute (positive) value of numbers.

For example, we could round the budget to millions of dollars:

Note that where the previous example created a new column by assigning to a column name that was not yet in the data, this directly modifies the existing budget column by re-assigning it to an existing name.

Chaining calculations and calculating multiple values

Note that you can calculate multiple columns at the same time. For example, you can compute the total gross and round both gross income and budget to millions of dollars:

Three things to note here. First, we assign the result of the mutate to a new object, bechdel_recoded, which means that this is a new copy of the data and the original bechdel will also still be available. Second, we added an extra digits=1 option to the round function. Lastly, note also that the second line computes gross, and the third line then modifies this newly computed value. In general, you can use the result of previous calculations directly within the same mutate function.

TipCommon error: Closing brackets and R/RStudio prompts

The example above included a multi-line mutate statement. More complex and longer statements will be quite common as your code becomes more sophisticated. One danger is that you may forget the closing parenthesis ()).

In the online interactive prompts, this will lead to a somewhat understandable error message: unexpected end of input – the input ended, but R was still expecting a closing bracket.

If you run an include chunk of code in Rstudio / R, however, it will not give an error message but instead you will see that the prompt changes from > to +:

> mutate(bechdel, budget = round(budget / 1000000, 1)
+

What the + means is that R is still waiting for you to close a parenthesis or a quoted text. You can give many extra commands after this, but it will patiently wait for you to close the parenthesis before actually running all of them.

So, when R feels unresponsive, check that it actually gives a > prompt. If it shows an unexpected + prompt, you can click to activate the console window, press Escape to cancel the current command, and fix and restart the command that was causing trouble.

Exercise

Starting from the original bechdel data set, modify the code below to compute two new columns:

• gross: The total gross income of a movie (international plus domestic)
• roi The Return on Investment of each movie (gross income divided by budget)

TipHint:

• You should compute two columns with two separate arguments to mutate
• Use the pattern mutate(data, new_column = calculation)
• Remember that you don’t need to quote column names, and that you can use the column created in the first calculation in the second

Important🎓 Solution:

• You should compute two columns with two separate arguments to mutate: mutate(bechdel, gross = ____, roi = ____)
• Use the pattern mutate(data, new_column = calculation, second_column = calculation)
• Remember that you don’t need to quote column names, and that you can use the gross column created in the first calculation in the second

bechdel <- mutate(bechdel, gross = foreign_gross + domestic_gross, roi = gross / budget ) head(bechdel)

bechdel <- mutate(bechdel,
  gross = foreign_gross + domestic_gross,
  roi = gross / budget
)
head(bechdel)

5.2 Recoding values

So far we’ve used mutate to calculate new values from old ones. Its other major job is recoding — turning the values in a column into different values. Recoding shows up constantly: collapsing many categories into a few, binning a continuous variable into ranges, or flipping cryptic codes into readable labels. R offers a few different functions for this, depending on what shape the recoding takes.

Recoding into two options with if_else

The if_else function checks a condition for each row, and chooses one value if it’s true, or an alternative value if it’s false. For example, we can check whether the gross income of a film was higher than its budget, and designate that it made a profit if so, and a loss otherwise:

Second, the syntax of if_else is if_else(condition, value_if_true, value_if_false). So, the first argument (gross > budget) tells R to check for each row whether the gross column is larger than budget. Then, if it is true the second argument will be chosen (profit), while for rows where it is false the third argument is chosen (loss)

An important things to note here is that the if_else function is used within mutate. This is generally true for the functions in this chapter: if_else computes a value, and mutate places that value in the data frame.

Recoding multiple values with recode_values and replace_values

Next, let’s consider the use case of recoding multiple values. For example, the bechdel data set contains a test column, with 10 different values: 5 possible test outcomes (nowomen, notalk, men, dubious, and ok), where each outcome can have -disagree appended to show that there was substantial disagreement among the (crowd) coders this column was based on.

We can use the recode_values function to recode each test value into a new value with a more descriptive name:

Please take a second to look at the slightly odd syntax of recode_values. Its first argument is the input column (test). This is followed by a series of arguments of the form old_value ~ new_value, where you can read the tilde (~) as a kind of arrow: old ⤳ new. So, you can specify as many recode rules as you need.

If multiple recode rules have the same target value, you don’t need to make a separate rule for each input. You can also specify more than one input value to replace by using the c() function to combine multiple values. For example, if we wish to drop the distinction between disputed and undisputed codings, we could do:

As briefly mentioned above, recode_values has a sister function replace_values. The difference between the two is what happens if you don’t specify all possible input values. In the case of replace_values, unnamed values are left unchanged. In contrast, recode_values will turn them into missing values (NA), unless an optional argument default="Some value" is given, in which case that value is used instead.

NotePackage versions and updating packages

In this section we used the recode_values function. This function was introduced in dplyr version 1.2, released in April 2026.

If you run this on your own computer (e.g. in RStudio) and get an error could not find function "recode_values", it’s possible that you have an older version of dplyr.

In general, to check what version of a package you have installed you can use:

> packageVersion("dplyr")
[1] ‘1.2.1’

If you have a version below 1.2, you should either upgrade it or use the older function case_match (which is very similar to recode_values). To upgrade a specific package, you can use the install.packages code that you probably already used to install tidyverse initially. You can also use update.packages(ask=FALSE) to update all packages that have an available update. However, it is a good idea to restart R before upgrading packages, as (especially on Windows) it can be problematic to upgrade a package that is already loaded. So, the procedure is:

1. Save your current script(s)
2. Restart the R session (Session → Restart R)
3. Update some or all packages: install.packages("dplyr", dependencies=TRUE) or update.packages(ask=FALSE)

After this, you can run packageVersion("dplyr") again to see whether the new version is installed.

Note that on restarting R, by default R will save and load your ‘environment’, that is the data you have currently loaded in memory. This feels convenient, but we strongly advise you to disable saving the environment. Go into Tools → Global Options, uncheck Restore .RData into workspace at startup and set Save workspace to .RData on exit to Never.

This can prevent problems with automatically re-loading packages on restarting R, but more importantly it is better ‘code hygiene’: you want the results of your script to only reflect the actual commands in the script, and not be dependent on data you have loaded in your environment.

Note that for more serious package management, advanced users might want to look at renv for per-project package management, and/or pak for more sophisticated dependency management.

Arbitrary recoding with case_when

recode_values allowed us to create recode rules by specifying old ⤳ new pairs. Sometimes, however, there is no clear 1:1 mapping between old and new, especially when dealing with continuous ranges or comparisons involving multiple columns.

For a simple example let’s start with the task of recoding the year into periods:

Again, take a minute to look at the syntax of case_when, as it is similar to, but also slightly different from recode_values:

1. First, case_when has no separate input column argument, jumping directly to the recode rules which include the column name(s) in their conditions.
2. Second, the recode rules take arbitrary conditions, such as the year < 1980 condition, rather than specifying only the input value. This makes the conditions a bit more verbose as you repeat the column name, but do offer the flexibility of using more powerful expressions, such as the comparisons used above.
3. Third, case_when will evaluate the rules in the order given. So, if year < 1980 it will return 1970, even if the next condition (year < 1990) also matches.
4. Finally, like recode_values it accepts a .default= argument which is used if none of the rules match, but confusingly it’s called .default= here, and default= for recode_values

Note that the example above shows only a relatively simple application of case_when (and frankly one that could be replaced easily by a recode_values). The real power of case_when lies in its ability to use multiple columns and conditions that cannot be simply expressed as discrete values.

For a more complicated example, let’s classify movies into big-budget success, big-budget flop, indie success, and indie flop:

Again, have a careful look at the code above. It features three new columns that all build on each other (gross, roi, and category). In the case_when rules, it combines comparisons of the budget and the roi:

1. First, a high budget and roi > 1 is coded as a ‘big-budget success’.
2. The next rule codes any other film with a high budget as a ‘big-budget flop’: a check for roi <= 1 is not needed as these would have been matched by the first rule.
3. Then, it codes every remaining profit-making film as an ‘indie success’, as all high budget films are already coded by the first two rules.
4. Finally, all remaining films must be ‘indie flops’, as all other categories are already coded.

TipWhen to use which recode function?

This section introduced three different recode functions, each with their own use case. Like a carpenter with different tools in their box, you should choose the most appropriate tool for a specific use case. To help, the table below summarizes the three functions:

Function	Description	Use case
if_else(cond, yes, no)	Returns yes where cond is TRUE, no otherwise	Binary outcomes (pass/fail, profit/loss)
recode_values(x, old ~ new, ...)	Replaces specific input values with specific output values	One-to-one or many-to-one value substitution for categorical data
case_when(cond ~ result, ..., .default = ...)	Evaluates rules top-to-bottom, returns the result of the first matching condition	Multi-way recoding based on conditions — ranges, thresholds, multi-column logic

R being R, there are of course many more functions for recoding. However, we suggest using the newer recode_values instead of recode or case_match (unless you’re stuck on an old R version), as the newer function is a clear improvement over the older functions. Similarly, avoid the built-in ifelse function in favor of tidyverse’s if_else (with the underscore _) function: the functions are generally the same, but if_else avoids some serious (but rare) problems with data types, for example when dealing with date/time values.

Exercise

To see whether films that pass the Bechdel test do better or worse financially, we need to compute a single column that determines whether it passed the test or not.

In the code below, starting from the original bechdel dataset, compute two columns: - roi: the total gross income divided by the budget - passed: Use if_else to determine whether a film passed the test or not. Treat both ok and ok-disagree as passing.

TipHint:

• For the roi, see the previous exercise. Can you combine it into a single computation?
• For if_else, use the pattern if_else(condition, "passed", "not passed")
• For the condition, you can use value %in% c(option1, option2)

Important🎓 Solution:

• For the roi, you can do roi = (domestic_gross + foreign_gross) / budget in a single line
• For if_else, use the condition test %in% c("ok", "ok-disagree")

bechdel <- mutate(bechdel, roi = (domestic_gross + foreign_gross) / budget, passed = if_else(test %in% c("ok", "ok-disagree"), "passed", "not passed") ) head(select(bechdel, year, title, roi, passed))

bechdel <- mutate(bechdel,
  roi = (domestic_gross + foreign_gross) / budget,
  passed = if_else(test %in% c("ok", "ok-disagree"), "passed", "not passed")
)
head(select(bechdel, year, title, roi, passed))

The code above computed the needed columns (roi and passed) – what would the next step be for answering the question whether there is a relation between the two? What data steps would you need to get to that answer?

More technically, our proposed solution for the question above was using if_else, but you could also use either recode_values or case_when to achieve the same result.

• Can you modify the code to use either function?
• Why do you think we used if_else here?
• Would you have done the same?

Finally, Chapter 9 will talk about dealing with text in more detail, but as a sneak preview the function str_remove(column, text_to_remove) will remove the specified text from the column. Can you use this to remove the -disagree label in the test column, and simplify the if_else function?

5.3 Grouping and summarizing data

So far, all transformations have worked on individual rows. For many questions, however, we want to compute values for groups of rows. For example, what is the average roi for films that pass the Bechdel test compared to films that don’t?

Summarizing data with group_by and summarize

Using the functions group_by() and summarize() we can answer this question quite directly:

The code above consists of two steps. In the first step, the data is grouped by passed. This doesn’t make any change in the data itself, but prepares for the next step by defining the two groups of interest.

The second step takes the grouped data (bechdel_grouped) and uses the summarize function. Syntactically, this function looks identical to the mutate function: you specify column = calculation, and it puts the result of the calculation in the named column.

The big difference is that summarize computes a single value for each group of rows: in this case, it computes the mean of the roi values (setting na.rm=TRUE to ignore any missing values).

As you can see from the result, summarize produces a radically smaller data frame than the original, which had 7 columns and almost 1,800 rows. In fact summarize will always create a data frame with one row per group (in this case, passed and not passed), and keeps only the grouping columns (passed) and the columns with the summary values (mean_roi).

Substantively, it seems that films that pass the Bechdel test actually have a lower return on investment than films that don’t. Surprisingly, this seems to go against the conclusions of the original article. We’ll dive into this in more detail later, but first we need to introduce a new piece of syntax.

Pipelines: The |> operator

The example above used two functions (group_by and summarize), and the output (result) of the first function was used as the input (first argument) of the next function. This essentially creates a ‘pipeline’ of operations that are applied to the data.

Since this is so common, R has introduced a special operator to make this more convenient: the pipeline operator |>. Let’s rewrite the code above with this new operator:

As you see, the two statements are now ‘chained’ together in a single pipeline: the bechdel data is piped into the group_by function, and the result of the grouping is piped into summarize.

The biggest advantage of this is that it removes the awkward intermediate objects bechdel_grouped, and it saves having to type the name of the data set for every function.

We introduce this notation with group_by() |> summarize() because these functions form a natural pair, but note that almost all tidyverse functions can be chained together in a pipe like this.

It is important to stress that there is absolutely no difference in the end whether you use the pipeline notation or use separate statements for each step. However, since data processing and analysis very often boils down to a sequence of steps (like read data, filtering it, computing values, and visualizing the result), pipelines are a very natural way to capture this. So, regardless of whether you use the notation, it is important that you are able to read it.

One last important thing to note is that |> was introduced as a base R feature in 2021. Before that, tidyverse shipped with its own pipeline operator, %>%. For all our concerns, the two operators are identical, and there is no reason to use the older operator anymore. However, you will find many examples on the Internet still use %>%, and probably as a result code generated by AI also tends to use it. So, if you see %>% being used just mentally translate to |>, and you should stick to |> in your own code.

Improving the film analysis pipeline

When we first calculated the mean ROI grouped by test outcome, we were surprised to find that passing the test was actually a negative predictor of return on investment, contrary to the claim made in the original 538 article. Also, if you are familiar with the film industry, you would also have noted that returns on investment in the 8-11 range are very atypical: most films are happy to make a small profit, but films earning many times their budget are quite exceptional (and statistically should probably be seen as outliers!)

To improve our analysis and get closer to the last figure of the article (Dollars Earned for Every Dollar Spent), we should make three changes to our analysis:

1. We should use the median rather than the mean ROI, which is much less sensitive to outliers
2. We should filter on films from 1990-2013 (as that is what they used)
3. We should categorize ‘dubious’ films as passing the test, and split between domestic and foreign ROI.

Using the pipeline syntax introduced above, we can make a single analysis pipeline that performs all these steps correctly:

This gives quite a different picture! The numbers are now much more realistic, with up to 37% median profit rather than the earlier exaggerated means. More importantly, the numbers now match the outcome of the 538 article: Films that pass the Bechdel test actually perform better in the US (and Canada, which is considered domestic), while performing similarly in the foreign market. This shows the importance of using the right summary statistic (as well as of preprocessing decisions).

Some things to note about this code:

• Similar to mutate, we create multiple columns in the summarize call by passing named arguments, n=, roi_domestic= and roi_foreign=
• We used n=n() to add the total amount of cases per group. This is a very useful function, but it can be confusing since it doesn’t take any arguments - it just counts the number of rows in each group.
• We don’t store the result in an object, but only display it on the screen. If you want to continue analysing the summary result, you can simply assign it to an object like any other.
• We use str_remove(test, "-disagree") to remove the -disagree suffix from the test column, see Chapter 9 for an in-depth discussion of text handling
• We prefix the outcomes with a number (1 to 4) to make the alphabetic sorting match the natural ordering. See Section 6.3.1 for a more elegant solution

Tip 5.1: Counting things: n(), tally() and count()

The code above used summarize(n=n()) to count how many rows each group contained. Since counting rows is such a frequent task, tidyverse includes multiple shortcuts for different use cases. The bullets below show how these different ways of counting can be used:

• n() is used within a group_by |> summarize and you need to assign it to a column yourself (e.g. summarize(n=n())).
• tally() is shorthand for summarize(n=n()). It is (generally) used after a group_by but comes instead of summarize, and automatically assigns the number of rows to a new column called n.
• count(columns) is shorthand for group_by(columns) |> summarize(n=n()) and so is used on its own to summarize a data frame to a count of rows within groups.
  
• add_count(columns) is similar to count, but adds a column with the count instead of replacing the data frame. It is shorthand for group_by(columns) |> mutate(n=n()) which will be explained in Section 5.3.6 below.

Thus, the three lines below all produce exactly the same output:

Grouping on multiple columns

The code above grouped by a single column, but you can also group by multiple columns.

For example, we could calculate the median budget and return-on-investment grouped by decade and whether the film passed the test:

As you can see, from the 1990s there was an increasing budget gap between films that passed and did not pass the Bechdel test, while the return on investment of the passing films was consistently higher.

TipMultiple grouping levels, ungroup(), .groups, and .by

You might have noticed a message from the code above that `summarise()` has regrouped the output. If you run it locally it would give more details:

`summarise()` has regrouped the output.
ℹ Summaries were computed grouped by decade and passed.
ℹ Output is grouped by decade.
ℹ Use `summarise(.groups = "drop_last")` to silence this message.
ℹ Use `summarise(.by = c(decade, passed))` for per-operation
  grouping instead.

This message is somewhat annoying, but also very informative.

First, after the operation Output is grouped by decade. This means that the resulting dataset is still grouped by decade: the default behavior of summarize with multiple grouping levels is to drop the last level (in this case passed). This can be useful if the next operation benefits from the decade grouping, but sometimes it is also unintended. You can always run ungroup() after the summarize to drop all grouping levels.

Next, you can explicitly set this behavior by specifying the .groups option, with "drop_last" being the default behavior, and .groups="drop" dropping all grouping levels after the summarize (so equivalent to adding ungroup after the summarize)

Finally, and perhaps most usefully, the message hints at a new feature introduced in 2023: defining the groups within the summarize using .by, removing the need for a preceding group_by as well as the confusion over remaining grouping levels.

Thus, in recent R versions the three calls below are identical*, and the .by version is the most compact and probably the most readable:

We primarily use group_by rather than .by= in this chapter because it is still the more commonly found construction, but in many cases you will find that .by is actually more convenient. In any case, either works fine so use what you find most natural.

*) Well, almost identical: the first call will emit the message as described before. Moreover, both group_by versions will reorder the output by the grouping levels, while the .by= version will leave the original ordering intact.

Exercise: Bechdel test over time (part 1)

The first figure in the 538 article shows a bar chart per (half) decade of the proportion of films passing the bechdel test. This exercise builds towards reproducing this chart by computing the total number of films per decade per bechdel outcome.

Concretely, starting from the bechdel_decade data:

• Use str_remove to remove the -disagree suffix
• Summarize the data to get the total amount of films per decade for each test category

TipHint:

• Use test = str_remove(test, ‘-disagree’) to remove the suffix (overwriting the column)
• Group by decade and test
• Use the n() function to count the number of films

Important🎓 Solution:

• Use test = str_remove(test, ‘-disagree’) to remove the suffix
• Group by decade and test
• Use the n() function to count the number of films

bechdel_over_time <- bechdel_decade |> # remove the `-disagree` suffix mutate(test = str_remove(test, '-disagree')) |> # summarize total amount of films per decade for each `test` category group_by(decade, test) |> summarize(n_films=n()) # Note: using .by instead of group_by would also be fine: # summarize(n_films=n(), .by=c(decade, test)) head(bechdel_over_time)

bechdel_over_time <- bechdel_decade |>
  # remove the `-disagree` suffix
  mutate(test = str_remove(test, '-disagree')) |>
  # summarize total amount of films per decade for each `test` category
  group_by(decade, test) |>
  summarize(n_films=n())
# Note: using .by instead of group_by would also be fine:
#   summarize(n_films=n(), .by=c(decade, test))
head(bechdel_over_time)

Adding group statistics with group_by |> mutate

Above we used group_by |> summarize, which replaced the current data set by a summary frame containing only one row per group and dropping all columns except for the group identifiers and summary columns.

Although group_by is often paired with summarize, you can also follow it with a mutate call.

For example, using the decade column calculated earlier, we can calculate the average budget of films per decade:

Two things to notice. First: rather than replacing the data frame with the summary values, group_by |> mutate added the summary values to the existing data set. Second, the values for mean_budget are repeated for the top six rows. In fact, the mean budget in that decade will be repeated for each row in each decade. In that sense, group_by |> mutate is very close to a normal mutate call, in the sense that it just adds (or replaces) the specified columns, leaving the rest of the data alone. What is special about the group_by is that any summary values used in the subsequent mutate are not calculated over the whole data set, but are calculated for each group: in this case the mean budget per decade.

It might seem strange to repeat a value like this, but it is really useful in cases where you want to compare individual rows to the group statistic. For example, we could see which films had a relatively high budget compared to other films in that decade:

As you can see, Avatar was the film with the most exceptionally high budget, having cost more than 8 times the average film in that decade. Interestingly, this top-6 of (relatively) exceptionally expensive movies is a decent list of well known blockbusters, with the possible exception of Shaft (1971) - but admittedly the authors of this book weren’t around back then.

Note that the list of relatively expensive movies is not saved as an object here – it is piped to head(), printing the top 6 rows on the screen for inspection without saving it.

Selecting items with slice_*

A final useful operation on groups is to slice them, for example to take the highest grossing film per decade:

• Do those films make sense to you?
• It seems odd that the 2010s had a relatively low highest grossing film. What could cause this? How can you check this in the data? (hint: try changing decade to year in the code above, and then adding |> print(n=100) after the select to print all rows)
• How do slice and group_by interact? What would happened if you slice without a group_by?

The code above takes a slice for each group, in this case taking the single (n=1) row with the highest gross within that decade. The with_ties=TRUE (which is also the default) means that in the unlikely case that two films had identical gross, it would keep both rather than only selecting the first - set it to FALSE to ensure you get only a single row per group.

Besides slice_max and its companion slice_min, you can also use slice_head or slice_tail to select the first or last row(s) in each group (especially useful if the data is already ordered); and slice_sample to take a random sample of rows from each group (useful for stratified samples).

Exercise: Bechdel test over time (part 2)

The first figure in the 538 article shows a bar chart per (half) decade of the proportion of films passing the bechdel test. This exercise builds on the summary made in the previous exercise, turning the absolute counts per decade into proportions.

Concretely, starting from the bechdel_over_time data, compute two new columns: - total_n: The total number of films in each decade - proportion: The percentage of films per category within each decade

The ggplot code to produce the stacked bar chart is given, but don’t worry if it doesn’t make sense yet - it will be explained in detail in the next chapter.

TipHint:

• Group by decade only, since you want to compute the total number of films per decade
• Use sum(n_films) to calculate the total number of films
• You can re-use the calculated total_n in the calculation of proportion

Important🎓 Solution:

• Use group_by(decade), since you want to compute the total number of films per decade
• Use total_n = sum(n_films) to calculate the total number of films
• Re-use the calculated total_n in the calculation of proportion

# Solution with blanks filled in: bechdel_over_time_rel <- bechdel_over_time |> group_by(decade) |> mutate(total_n = sum(n_films), proportion = n_films / total_n) # No need to change the code below, but feel free to have a look! ggplot(bechdel_over_time_rel, aes(x=decade, y=proportion, fill=test)) + geom_col(position="stack") + theme_classic() + scale_y_continuous(labels = scales::percent) + labs(title="The Bechdel Test Over Time", x = "Decade", y="", caption="Source: github.com/fivethirtyeight/data")

# Solution with blanks filled in:
bechdel_over_time_rel <- bechdel_over_time |>
  group_by(decade) |>
  mutate(total_n = sum(n_films),
         proportion = n_films / total_n)

# No need to change the code below, but feel free to have a look!
ggplot(bechdel_over_time_rel, aes(x=decade, y=proportion, fill=test)) +
  geom_col(position="stack") +
  theme_classic() +
  scale_y_continuous(labels = scales::percent) +
  labs(title="The Bechdel Test Over Time",
      x = "Decade", y="", caption="Source: github.com/fivethirtyeight/data")

In this chapter we got two quite different answers to question whether “do Bechdel-passing films make more money?”, depending on seemingly small methodological choices:

• mean vs median
• what counts as “passing”
• which years we include
• domestic vs worldwide gross

Which choice do you think is the most impactful? Can you re-run the analysis with only that choice made differently? How does your conclusion change?

Now imagine a journalist asks you: “Do Bechdel-passing films perform better financially?” What would you say?

5.4 Column types: texts, numbers, and dates

In R, every object and column has a data type, for example character (for textual data) or numeric (confusingly abbreviated to <dbl> because internally it’s stored with ‘double’ precision). This type is very important in determining how data is handled and what operations are possible. For example, you can add numbers together but not texts.

In most cases, R will automatically detect the type when you load a file. For example, read_csv reads the first 1,000 rows of a file and then guesses the right column type.

In some cases, however, you might want to change this yourself. Consider the following dataset of US county-level demographics:

As you can see, this has a 5-digit fips (Federal Information Processing Standards) code identifying each county. In this case, it is ingested as a character (<chr>) column because it starts with a 0 (which would be lost as a number, since 01001 and 1001 are the same numerical value).

Now, you could change this default in read_csv: try adding col_types = list(fips="numeric") as an argument to read_csv in the code above, and checking whether the column type changed to numeric (<dbl>).

However, you can also change column types in a mutate call. For example, the following will change the character column to a numeric column:

Similarly, you could use as.character to turn it back into a textual column:

However, this of course does not bring back the leading zero. For this, we can use the str_pad function to ‘pad’ a ‘string’ to a given length. Try replacing as.character in the code above with str_pad(fips, width=5, pad = "0"). Note that str_pad is part of the stringr package for dealing with textual data, which will be covered in Chapter 9 for more details.

Formatting numbers

Now suppose we want to display the column nonwhite_pct as a textual percentage. We could use as.character to convert it to a character column:

However, this mostly shows that the number is internally stored with way more decimals than we will ever need (and probably also more than warranted by the actual measurement precision). We could use the format function to set the number of digits:

Interestingly, this works but gives a single decimal digit for the percentages. The reason for that is that digits specifies the minimum number of significant digits, rather than the number of decimal places.

The easiest way to achieve a fixed number of decimal places is to use the number() function from the scales package, which is included when you installed tidyverse but not loaded by default:

5.5 Working with Dates and Times

Many data sets in social science research have a temporal dimension: (social) media coverage of a campaign or issue changing over time; yearly or even daily economic indicators; panel survey responses per week or per year.

Working with such data requires handling date or date+time columns. For example, let’s look at a dataset of Bluesky posts around the Brazilian 2024 elections (see the book’s github repository for the R code used to scrape these posts):

As you can see, this dataset includes a row for every post, which have a created_at and indexed_at column, representing the timestamp they were created by the user’s client and indexed by the bluesky server, respectively. Both columns are automatically recognized and parsed as timestamps (dttm means date and time).

Extracting parts of a date

When working with dates or timestamps, you commonly want to do things such as calculating the day of week, or grouping events by day, week, or month. This is made very easy with functions from the lubridate package (see the cheat sheet). For example, we can use wday to calculate the day of the week for each post, and see on which days people posted the most:

Other extraction functions include date (to strip the time part), year, quarter, month, etc.

Besides using functions to extract part of a date, you can also subtract two dates to calculate the difference. As noted, the posts data frame has an indexed_at column as well as a created_at column.

• Can you compute a new column to calculate this difference? What does this number mean substantively?
• What is the class() of this column? What happens if you convert it to numeric?
• as.numeric has a units= option for time differences. What changes if you set it to units="mins"? (note, see ?difftime for more information on this)
• Finally, can you make a histogram of time differences. Do they make sense?

You can use the empty chunk below to test and run code:

Grouping dates and timestamps with floor_date

Besides the extraction functions mentioned above, a very useful function is floor_date. This function ‘rounds down’ a date to the start of a period such as year, quarter, day, or hour. This allows you to easily group posts (or other items) that were created in the same quarter or day. For example, the code below shows the number of posts per month:

Note that rounding down to the month is quite different from extracting the month: month(as.Date("2024-04-17")) will return 4 (the month number), while floor_date(as.Date("2024-04-17"), unit="month") will return “2024-04-01”, i.e. the first date of that particular month. So, if you want to know if people post more in the evening or in the morning, you want to extract the hour using hour(), since that will pool all posts at the same hour, regardless of the day. If you want to make an hourly timeline of posts over multiple days, you should use floor_date(unit="hour"), which returns distinct hours for each day.

NoteOn week numbers

For some applications, weeks are the most logical unit of analysis. Many activities, including work, leisure, and (social) media production and consumption follow weekly patterns. When aggregating per month, for example, some months will have four weekends while other months have five, which can make a big difference for looking at sporting, dating, but also religious activities.

A possible issue here is that years don’t always start on a Monday, so weeks can be spread over two calendar years, but of course we would like every week to have seven days. Using floor_date on weeks is safe, as every date will be ‘rounded down’ to the previous Sunday (or Monday, if you set week_start=1). Wednesday, January 1st will simply be changed to Sunday 29 December of the previous year.

When calculating the week number for a given date, however, it becomes more complicated. The week() function will simply start the first week on January first running to January 7, regardless of the day of week. Although simple, this means that the weeks won’t always start on a Monday (or Sunday), which is generally not how we think of weeks.

The most standardized solution is to use ISO weeks, named after the ISO 8601 standard which defines week 1 of a year as the first Monday-Sunday week with at least four days in it. So, if January 1st is a Friday (or Saturday or Sunday), it will be seen as part of the last week of the previous year, while if the year starts on a Thursday (or Monday-Wednesday) this will be week 1, and the last days of the previous year are also seen as part of that week.

The advantage of this is that every week is a full Monday-Sunday week, but the price is that some days belong to a week from a different calendar year. For this reason, the isoyear function provides the year that corresponds to the week number, so for Friday January 1st the isoyear will be the previous calendar year.

To see this in action, the following code shows the isoyear and isoweek for the last and first day of two years:

Note that while floor_date(unit="week") by default assumes a week starting on Sunday, the ISO standard defines weeks as starting on Monday. If this is a problem, you can use floor_date(unit="week", week_start=1) to tell floor_date to start on Mondays. Alternatively, there is a related pair of functions, epiweek and epiyear, which give the epidemiological week: this is a very similar standard to isoweek, but starts the week on Sunday rather than Monday.

Parsing and formatting dates

In the example above, the timestamps were automatically recognized as such and parsed by R, either implicitly (e.g. when reading data in read_csv) or explicitly (e.g. in as.Date("2024-01-01")) The reason for this is that they were all given in the unambiguous ISO standard notation: "2024-04-17" for a simple date, or "2024-12-16T18:42:03Z" for a timestamp. In general, this is the preferred format for dates and times: First, it is not ambiguous in the way 4/7/76 is (is it fourth of July or April 7? And how sure are you that it’s 1976?). Additionally, since it goes from largest to smallest unit, it sorts properly using alphabetic sorting.

However, in many cases we will have to use data which is produced with a national standard, and we might want to label graphs with dates that are more readable to a specific audience.

To format dates, the best option is the format function:

As you can see, the function takes a format parameter. This parameter gives the template to be used for formatting the dates, and can include literal characters (such as , or -) as well as %-placeholders for the year, month, et cetera. The table below gives a list of the most commonly used placeholders:

Table 5.1: Common date and time format codes

Code	Meaning	Example (for 2024-04-07 18:42:03)
%Y	Year, four digits	2024
%y	Year, two digits	24
%m	Month number	04
%B	Month name	April
%b	Month name, abbreviated	Apr
%d	Day of the month	07
%A	Weekday name	Sunday
%a	Weekday, abbreviated	Sun
%H	Hour (24-hour clock)	18
%M	Minute	42
%S	Second	03

To parse dates, the easiest option is to use the parsing functions that are shipped with lubridate. These functions tell R the order of fields to read, and R takes care of the rest. So, to parse the prettydate column we created earlier, we use the dmy function since the field is formatted day-month-year:

As you can see, this re-parses the date part of the original timestamp and deals with the fact that the month is given as a name automatically (but see the note below on languages for month and week names). Similar functions are included for parsing timestamps. The example below shows how the date is formatted using American m/d/y notation with time included, and then parsed using mdy_hm (month-day-year_hour-minute):

Time zones

As a final complication, timestamps in R are always associated with a time zone. In the CSV file we used for the Brazilian Bluesky posts, this was explicitly given at the end of the timestamp: the Z in 2024-12-16T18:42:03Z means the ‘plus zero’ time zone, that is UTC or Greenwich Mean Time (also known as Zulu time, after the NATO alphabet pronunciation of the letter Z). You can check the current time zone of the timestamp column by calling tz(posts$created_at).

Of course, since most of the posts will be from Brazil, it might be more appropriate to set it to Brazilian time using the with_tz function. For example, the code below adds a version of the timestamp in Brazilian time, and shows a line plot of hourly posts in both time zones:

As you can see, the Brasilia time zone is 3 hours behind UTC, placing most posts between 9:00 and 22:00 rather than the less plausible noon to after midnight.

This example works because the data was explicitly given in the UTC time zone. Often, however, you might read data that implicitly assumes a local time zone. In that case, you should tell R what the time zone is by using e.g. timestamp=force_tz(timestamp, "Europe/Amsterdam"). So, use with_tz to change the time zone on a column that already has a correct time zone, and force_tz to tell R that the column is actually in a different time zone.

Dates in graphs

Although we will discuss visualization in more detail in Chapter 6 and Chapter 8, it is useful to know that ggplot handles dates and timestamps well by default. This means that it will automatically use sensible names to place on an axis. For example, when plotting the number of posts per week it will decide that given the specific period, quarterly ticks make most sense:

Note that you can change this default using scales: try adding + scale_x_datetime(date_labels="%G-%V", date_breaks="4 weeks") to the code above (and see Chapter 8 for a more in-depth explanation of ggplot scales)

NoteMonth and weekday names and locales

When dealing with month or weekday names, the results will of course be language-specific. By default, the formatting and parsing functions used above will use the local settings of the machine the code is run on. You might have noticed that the graph above uses English month names in the browser. If you run it on your own computer, it might use another language depending on your local settings.

If you only work with the language matching your local settings, this is probably fine: R will automatically parse and format dates using the right month or day names.

However, if you are for example producing an English language report on a French-language computer, you will probably prefer English names for the months. You can change the language settings in R using the Sys.setlocale command, which takes a locale identifier consisting of the language, region, and encoding. For example, Sys.setlocale("LC_TIME", "nl_NL.UTF-8") will set the date and time handling to Dutch, while Sys.setlocale("LC_TIME", "en_US.UTF-8") will set it to US English. Note that this setting will affect all subsequent R commands in this session, and the locale does need to be installed on your system (and in the interactive examples it does not do anything). Finally, locale strings are unfortunately different for different operating systems, so on Windows it will be called Dutch_Netherlands or English_United States instead.

For parsing dates that use names in a different language from the computer’s default, it is also possible to use the parse_date and parse_datetime functions from the readr package (included in tidyverse). These functions take a format string with the same %-placeholders as discussed above, as well as a locale= argument:

readr ships with month and day names for almost 200 languages, so should work for most languages, including e.g. French and Swahili but also globally less relevant languages like my beloved Dutch.

Exercise: Hourly posts on election day

The first round of the Brazilian municipal elections was held on Sunday 6 October 2024. Compute the number of posts per hour on the election date, making sure to use Brazilian time ("America/Sao_Paulo"). Note: To make it easier to check your results, the posts data frame here only contains the created_at and text columns.

TipHint:

Convert created_at to Brazilian time before filtering and extracting the hour. Which function changes the time zone of a timestamp without changing the moment it refers to? Should you extract the hour, or round down to the hour? Or would either work?

Important🎓 Solution:

Use with_tz(created_at, "America/Sao_Paulo") to show the timestamps in Brazilian time, and only then filter to election day. (if you would filter first, the filter would be based on the date in UTC, which might be different).

Since it’s a single day, you can use either hour() to extract the hour (0-23), or floor_date(unit='hours') to round down to the nearest hour (e.g. 2024-10-06 19:00).

posts |> mutate(date_br = with_tz(created_at, "America/Sao_Paulo"), hour = floor_date(date_br, unit="hours")) |> filter(date(date_br) == "2024-10-06") |> count(hour)

posts |>
  mutate(date_br = with_tz(created_at, "America/Sao_Paulo"),
         hour = floor_date(date_br, unit="hours")) |>
  filter(date(date_br) == "2024-10-06") |>
  count(hour)

Note that there are many ways to skin this cat: - You could calculate the hour after the filtering instead of before - As noted above, you could use hour() instead of floor_date(unit="hours") to extract the hour (but this might make it harder to see what goes wrong if the filtering is incorrect as hours from a different date gets silently included) - You could add an explicit as.Date("2024-10-06") (and some would probably say you should) - You can use group_by(hour) followed by either tally() or summarize(n=n()) instead of using the count shorthand

All of these are fine choices, this is just how I chose to solve it.

One of the wrong ways to ‘solve’ this exercise is the code below:

Look at the dates in the hour column. Unless you happen to be in the Brazilian time zone (i.e. In Brazil or a neighbour or possibly the easternmost parts of Canada), this should contain the wrong results.

(If you do happen to be in Brazil or Nova Scotia, or you just want to play around, you can change the system time zone by adding e.g. Sys.setenv(TZ="Europe/Amsterdam") at the top of the code. You can also add Sys.setenv(TZ="America/Sao_Paulo") to get the results for a reader from that time zone. Note that if you run this on your computer, the setting will persist until you set another time zone or you restart R)

• What could cause this? What do you think happens behind the scenes?
• How could you fix it within this format of comparing the full timestamp to a lower and upper range?

Quiz questions

Section 5.1 quiz: changing or computing values with mutate

Q1: Where did my column go?

A student has this data frame films:

title	budget	domestic_gross	foreign_gross
Frozen	150000000	400738009	876300000
Gravity	100000000	274092705	442981000

They run mutate(films, gross = domestic_gross + foreign_gross), see the new gross column printed, and then run head(films). But there is no gross column in the output. Why?

• The result of mutate was printed but not stored: they need films <- mutate(films, …).
• A mutate can only change existing columns; to add a new column you need a different function.
• The new column name should be quoted: “gross” = domestic_gross + foreign_gross.
• head() only shows the columns that were in the original data.

Q2: Reusing a name

Given the same films data as above, what does films <- mutate(films, budget = round(budget / 1000000, 1)) do?

• It replaces the values in the existing budget column with the budget in millions (150 and 100).
• It adds a second budget column next to the original one, so you can compare them.
• It gives an error, because budget already exists.
• It adds a new column called round(budget / 1000000, 1).

Q3: Building on a new column

Given this data frame films (amounts in millions of dollars):

title	budget	domestic_gross	foreign_gross
Little Women	40	120	80

What is the value of roi after running this code?

mutate(films,
  gross = domestic_gross + foreign_gross,
  roi = gross / budget)

• 5
• An error object ‘gross’ not found, because gross only exists after the mutate has finished.
• 3
• NA, because gross doesn’t have a value yet when roi is computed.

Section 5.2 quiz: recoding values

Q1: Age groups

You have survey data with the age of each respondent:

respondent	age	vote
1	23	GL-PvdA
2	67	PVV
3	41	VVD
4	35	D66

You want to recode age into three groups: “18-34”, “35-54”, and “55+”. Which function is the best fit?

• case_when
• recode_values
• if_else
• replace_values

Q2: Merging two parties

In 2023, the Dutch parties GroenLinks (GL) and PvdA formed a joint list. To compare earlier vote intentions with later ones, you want to treat them as a single party in this data frame d:

respondent	party
1	VVD
2	PvdA
3	GL
4	D66

Which command replaces both “GL” and “PvdA” by “GL/PvdA”, while leaving the other parties unchanged?

• mutate(d, party = replace_values(party, c(“GL”, “PvdA”) ~ “GL/PvdA”))
• mutate(d, party = recode_values(party, c(“GL”, “PvdA”) ~ “GL/PvdA”))
• mutate(d, party = if_else(party == “GL”, “GL/PvdA”, party))
• mutate(d, party = replace_values(party, “GL/PvdA” ~ c(“GL”, “PvdA”)))

Q3: The order of the rules

A student wants to recode the release year of films into decades:

title	year
Back to the Future	1985
Titanic	1997
Avatar	2009

mutate(films, decade = case_when(
  year < 2000 ~ "1990s",
  year < 1990 ~ "1980s",
  .default = "2000s"))

What will decade be for Back to the Future (1985)?

• “1990s”
• “1980s”
• “2000s”
• An error, because 1985 matches two different rules.

Q4: Three ways to recode

Using the bechdel data from this chapter:

title	test	budget	roi
21 & Over	notalk	13000000	3.24
Dredd 3D	ok-disagree	45000000	0.93
12 Years a Slave	notalk-disagree	20000000	8.34

Which of these statements about if_else, recode_values, and case_when is false?

• Just like recode_values, case_when takes the input column as its first argument, e.g. case_when(test, …).
• Recoding roi into “profit” and “loss” can be done with if_else, because there are only two outcomes.
• A rule that looks at both budget and roi at the same time is possible in case_when, but not in recode_values.
• The value for rows that match none of the rules is set with default= in recode_values, but with .default= in case_when.

Section 5.3 quiz: grouping and summarizing data

Q1: The shape of a summary

Given this data frame survey with trust in parliament (0-10):

respondent	country	trust
1	Netherlands	6
2	Netherlands	4
3	Hungary	3
4	Hungary	5
5	Hungary	4

What is the result of survey |> group_by(country) |> summarize(mean_trust = mean(trust))?

• Two rows, with only the columns country and mean_trust.
• Five rows, with a new column mean_trust that repeats the country mean (5 or 4) on every row.
• One row, with the overall mean trust (4.4).
• Two rows, with the columns respondent, country, trust, and mean_trust.

Q2: Compared to their compatriots

Using the same survey data, you want to know for each respondent how much their trust differs from the average in their own country. So respondent 1 should get 1 (6 minus the Dutch mean of 5), and respondent 3 should get -1 (3 minus the Hungarian mean of 4). Which code does that?

• survey |> group_by(country) |>
mutate(diff = trust - mean(trust))
• survey |> mutate(diff = trust - mean(trust))
• survey |> group_by(country) |>
summarize(mean_trust = mean(trust)) |>
mutate(diff = trust - mean_trust)
• survey |> group_by(respondent) |>
mutate(diff = trust - mean(trust))

Q3: From steps to a pipeline

Given the bechdel data with a computed passed column:

title	year	passed
21 & Over	2013	not passed
Dredd 3D	2012	passed
12 Years a Slave	2013	not passed

Your classmate used this code to compute the number of films per passed status:

bechdel_grouped <- group_by(bechdel, passed)
bechdel_counts <- summarize(bechdel_grouped, n = n())

How can you rewrite this as a single pipeline?

• bechdel_counts <- bechdel |>
group_by(passed) |>
summarize(n = n())
• bechdel_counts <- bechdel |>
summarize(n = n()) |>
group_by(passed)
• bechdel_counts <- bechdel |>
group_by(passed) |>
summarize(n = n(passed))
• bechdel_counts <- group_by(passed) |>
summarize(bechdel, n = n())

Q4: Mean or median?

Suppose these are the returns on investment (roi) of five films that pass the Bechdel test:

title	roi
Film A	0.5
Film B	1.1
Film C	1.3
Film D	1.4
Film E	40.0

The mean ROI is 8.86, and the median is 1.3. Why does the chapter switch from mean to median when comparing films?

• A single extreme hit pulls the mean far up; the median (1.3) is much closer to what a typical film earns.
• The mean function can’t deal with missing values, while median can.
• Only median can be used inside summarize; mean only works in mutate.
• Film E must be an error in the data, and median automatically removes such errors.

Q5: The biggest hit per decade

Given this data frame films (gross in millions of dollars):

title	decade	gross
Star Wars	1970	775
Jaws	1970	470
Titanic	1990	2187
Jurassic Park	1990	1029

A student wants the highest grossing film of each decade, but forgets the group_by and runs films |> slice_max(order_by = gross, n = 1). What is the result?

• Only one row: Titanic, the highest grossing film overall.
• Two rows: Star Wars and Titanic, because the data has a decade column.
• An error, because slice_max needs grouped data.
• All four rows, sorted from highest to lowest gross.

Section 5.4 quiz: column types

Q1: There and back again

Given this excerpt of the counties data, where fips is a character column:

fips	state	county
01001	Alabama	Autauga
01003	Alabama	Baldwin

What is the value of fips for Autauga after running counties |> mutate(fips = as.numeric(fips)) |> mutate(fips = as.character(fips))?

• “1001”
• “01001”, because converting back to character restores the original text.
• NA, because “01001” can’t be converted to a number.
• “1.001e3”, because R stores numbers in scientific notation.

Q2: Percentages for a table

For a report, you want to show the nonwhite_pct of each county as text with exactly two decimals and a percent sign, like “29.38%”:

county	nonwhite_pct
Autauga	29.377616894923854
Baldwin	19.257132908753015

Which call gives you that?

• number(nonwhite_pct, accuracy=.01, suffix=“%”)
• format(nonwhite_pct, digits=2)
• round(nonwhite_pct, 2)
• as.character(nonwhite_pct)

Q3: Numbers that aren’t numbers

A student reads a CSV file with election results, and gets this data frame:

municipality	turnout	votes
Amsterdam	78.3	“412 345”
Utrecht	83.1	“226 981”

Because of the spaces in the numbers, votes was read as a character column (<chr>). What happens if they run mutate(results, votes_k = votes / 1000)?

• An error (non-numeric argument to binary operator), because you can’t divide a text by a number.
• It works: R automatically converts the texts to numbers, giving 412.345 and 226.981.
• It gives NA for both rows, because the texts contain spaces.
• The calculation is skipped, and votes_k is simply not created.

Section 5.5 quiz: working with dates and times

Q1: Morning or evening?

Given these Bluesky posts (times in Brazilian time):

created_at	text
2024-10-05 09:12:44	Amanhã é dia de votar!
2024-10-06 21:03:10	Resultados saindo…
2024-10-07 09:40:02	Segundo turno em SP

You want to know whether people post more in the morning or in the evening, pooling all days together. Which column should you group on?

• hour(created_at)
• floor_date(created_at, unit=“hour”)
• wday(created_at, label=TRUE)
• date(created_at)

Q2: Rounding down a date

The first round of the 2024 Brazilian municipal elections was held on October 6. What does floor_date(as.Date(“2024-10-06”), unit=“month”) return?

• 2024-10-01
• 10
• 2024-10-31
• “October”

Q3: A Brazilian timestamp

You scraped a Brazilian news site that gives the time of each article in the usual Brazilian notation (day/month/year):

headline	published
Boulos e Nunes vão ao segundo turno	06/10/2024 21:15

Which function correctly turns published into a timestamp for 6 October 2024, 21:15?

• dmy_hm(published)
• mdy_hm(published)
• ymd_hms(published)
• as.Date(published)

Q4: The wrong time zone

You have an online survey where the start time of each response was recorded in Dutch local time, but without a time zone. When you read it, R assumes it’s UTC:

respondent	started
1	2024-03-01 09:00:00 UTC

The respondent actually started at 9:00 in the morning, Amsterdam time. How do you fix the started column?

• force_tz(started, “Europe/Amsterdam”)
• with_tz(started, “Europe/Amsterdam”)
• format(started, format=“%H:%M”), so the time zone is no longer shown.
• Nothing: it says 09:00, which is the correct time.

Q5: A readable date

Given a post with created_at equal to 2024-10-06 18:42:03, what does format(created_at, format=“%d %B %Y”) return (on a computer with English language settings)?

• “06 October 2024”
• “06 10 2024”
• “October 06 2024”
• “Sunday 06 2024”