lab_dataframes.Rmd

---
title: "Matrices, Lists and Dataframes"
subtitle: "R Programming Foundations for Data Analysis"
output:
  bookdown::html_document2:
    highlight: textmate
    toc: true
    toc_float:
      collapsed: true
      smooth_scroll: true
      print: false
    toc_depth: 4
    number_sections: true
    df_print: default
    code_folding: none
    self_contained: false
    keep_md: false
    encoding: 'UTF-8'
    css: "assets/lab.css"
    include:
      after_body: assets/footer-lab.html
---

```{r,child="assets/header-lab.Rmd"}
```

# Introduction

A data set that has more than one dimension is conceptually hard to
store as a vector. For these two-dimensional data sets the solution is to instead use matrices or data frames. As with vectors, all values
in a matrix have to be of the same type (e.g. you cannot mix characters and numerics in the same matrix. I mean, you can, but they will get coerced to characters). For data frames this homogeneity is not a requirement and different columns can have different data types, but all columns in a data frame must have the same number of entries. In addition to these, R also have objects named lists that can store any type of data set and are not restricted by types or dimensions.

In this exercise you will learn how to:

- Create and work with matrices, data frames and lists  
- Perform basic math on matrices  
- Use functions to summarize information from data frames  
- Extract subsets of data from matrices, data frames and lists  


# Matrices

The command to create a matrix in R is `matrix()`. As input it takes a vector of values, the number of rows and/or the number of columns.

```{r}
X <- matrix(1:12, nrow = 4, ncol = 3)
X
```

Note that if you only specify the number of rows or the number of columns, but not both, R will infer the size of the matrix automatically using the size of the input vector and the option given. The default way of filling the matrix is column-wise, so the first values from the vector ends up in column 1 of the matrix. If you instead wants to fill the matrix row by row you can set the byrow flag to TRUE.

```{r}
X <- matrix(1:12, nrow = 4, ncol = 3, byrow = TRUE)
X
```

Subsetting a matrix is done the same way as for vectors, but you have
two dimensions to specify. So you specify the rows and
columns you want.

```{r}
X[1,2]
```

If one wants all values in a column or a row this can be specified by
leaving the other dimension empty. For e.g. this code will print all
values in the second column.

```{r}
X[,2]
```

Note that if the retrieved part of a matrix can be represented as a
vector (e.g. has a single dimension) R will convert it
to a vector. Otherwise it will still be a matrix.

## Exercise

<i class="fas fa-clipboard-list"></i> Create a matrix containing the numbers 1 through 12 with 4 rows and 3 columns, similar to the matrix X shown above.

1.  How do you find out the length of the matrix?

```{r,accordion=TRUE}
length(X)
```

2.  Extract/subset all the values in the matrix that are larger than 6.

```{r,accordion=TRUE}
X[X>6]
```

3.  Swap the positions of column 1 and 3 in the matrix X

```{r,accordion=TRUE}
X[,c(3,2,1)]
```

4.  We can use `rbind` to add rows to a matrix, or `cbind` to add columns. How would you add a vector with three zeros as a fifth row to the matrix?

```{r,accordion=TRUE}
X.2 <- rbind(X, rep(0, 3))
X.2
```

5.  Replace all values in the first two columns in your matrix with `NA`.

```{r,accordion=TRUE}
X[,1:2] <- NA
X
```

6.  Replace all values in the matrix with 0 and convert it to a vector

```{r,accordion=TRUE}
X[] <- 0
as.vector(X)
```

7.  In the the vector exercises, you created a vector with the names of the type Geno\_a\_1, Geno\_a\_2, Geno\_a\_3, Geno\_b\_1, Geno\_b\_2&#x2026;, Geno\_s\_3 using vectors. We have previously mentioned a function named `outer()` that generates matrices based on the combination of two datasets. Try to generate the same vector as before, but this time using `outer()`. This function is very powerful, but can be hard to wrap your head around, so try to follow the logic, perhaps by creating a simple example to start with.

```{r, accordion=TRUE}
letnum <- outer(paste("Geno",letters[1:19], sep = "_"), 1:3, paste, sep = "_")
class(letnum)
sort(as.vector(letnum))
```

8.  Create two different 2 by 2 matrices named A and B. **A** should contain the values 1-4 and **B** the values 5-8. Try out the following commands and by looking at the results see if you can figure out what is going on.

```
A. A * B
B. A / B
C. A + B
D. A - B
E. A == B
```

```{r,accordion=TRUE}
A <- matrix(1:4, ncol = 2, nrow = 2)
B <- matrix(5:8, ncol = 2, nrow = 2)
A
B

A * B
A / B
A %x% B
A + B
A - B
A == B
```

9. Generate a 10 by 10 matrix with random numbers. Add row and column names and calculate mean and median over rows and save these in a new matrix.

```{r,accordion=TRUE}
e <- rnorm(n = 100)
E <- matrix(e, nrow = 10, ncol = 10)
colnames(E) <- LETTERS[1:10]
rownames(E) <- colnames(E)
E.means <- rowMeans(E)
E.medians <- apply(E, MARGIN = 1, median)
E.mm <- rbind(E.means, E.medians)
E.mm
```

# Dataframes

Even though vectors are the basic data structures of R, data frames are very central as they are the most common way to import data into R (e.g. `read.table()` will create a data frame). A data frame consists of a set of equally long vectors. As data frames can contain several different data types the command `str()` is very useful to get an overview of data frames.

```{r}
vector1 <- 1:10
vector2 <- letters[1:10]
vector3 <- rnorm(10, sd = 10)
dfr <- data.frame(vector1, vector2, vector3)
str(dfr)
```

In the above example, we can see that the dataframe **dfr** contains 10 observations for three variables that all have different classes, column 1 is an integer vector, column 2 a character vector, and column 3 a numeric vector.

## Exercise

1. Figure out how many columns and rows are present in the data frame.

```{r,accordion=TRUE}
dim(dfr)
# or
ncol(dfr)
nrow(dfr)
```

2. One can select columns from a data frame using either the name or the position. Use both methods to print the last two columns from the **dfr** data frame.

```{r,accordion=TRUE}
dfr[,2:3]
dfr[,c("vector2", "vector3")]
```

3. Print all letters in the **vector2** column of the data frame where the **vector3** column has a positive value.

```{r,accordion=TRUE}
dfr[dfr$vector3>0,2]
dfr$vector2[dfr$vector3>0]
```

4. Create a new vector combining all columns of **dfr** and separate them by a underscore.

```{r,accordion=TRUE}
paste(dfr$vector1, dfr$vector2, dfr$vector3, sep = "_")
```

5. There is a data frame of car information that comes with the base installation of R. Have a look at this data by typing `mtcars`. How many rows and columns does it have?

```{r,accordion=TRUE}
dim(mtcars)
ncol(mtcars)
nrow(mtcars)
```

6. Re-arrange (shuffle) the **row names** of this data frame and save the result as a vector.

```{r,accordion=TRUE}
car.names <- sample(row.names(mtcars))
```

7. Create a data frame containing the vector from the previous question and two vectors with random numbers named random1 and random2.

```{r,accordion=TRUE}
random1 <- rnorm(length(car.names))
random2 <- rnorm(length(car.names))
mtcars2 <- data.frame(car.names, random1, random2)
mtcars2
```

8. Now you have two data frames that both contains information on a set of cars. A collaborator asks you to create a new data frame with all this information combined. Create a merged data frame ensuring that rows match correctly.

```{r,accordion=TRUE}
mt.merged <- merge(mtcars, mtcars2, by.x = "row.names", by.y = "car.names")
mt.merged
```

9. Calculate the mean value for the two columns that you added to the **mtcars** data frame. Check out the function `colMeans()`.

```{r,accordion=TRUE}
colMeans(mtcars2[, c("random1", "random2")])
```

# Lists

The last data structure that we will explore are **lists**, which are very flexible data structures. Lists can combine elements of different types and they do not have to be of equal dimensions. The elements of a list can be pretty much anything, including vectors, matrices, data frames, and even other lists.  The drawback with a flexible structure is that it requires a bit more work to interact with.

The syntax to create a list is similar to creation of the other data structures in R.

```{r}
l <- list(1, 2, 3)
```

As with the data frames the `str()` command is very useful for the sometimes fairly complex lists instances.

```{r}
str(l)
```

This example of a list containing only a numeric vector is not very exciting, so let's create a more complex example.

```{r}
vec1 <- letters
vec2 <- 1:4
mat1 <- matrix(1:100, nrow = 5)
df1 <- as.data.frame(cbind(10:1, 91:100))
mylist <- list(vec1, vec2, mat1, df1, l)
```

As you can see a list can not only contain other data structures, but can also contain other lists.

Looking at the `str()` command reveals much of the details of a list

```{r}
str(mylist)
```

With this more complex object, subsetting/selecting is slightly trickier than with the other more homogeneous objects we have looked at so far.

You can think of the R **List** as a **Pea Pod**.

* Each **individual pea** inside the pod represents one **element** of the list.
* The **position** of the pea (1st, 2nd, 3rd) is its **index**.
* Any **label** on a pea (e.g., "Sweet") is the element's **name**. Just like we can name vector elements.

The core of list subsetting is understanding the difference between the two operators, `[]` and `[[]]`. The single square bracket operator `[]` is like using a **kitchen knife to cut a piece off the pod**.

`[]` always returns a **new, smaller pea pod** (a **new list**) containing the elements you selected. The integrity of the container is preserved.

| R Code | Analogy | Result |
| :--- | :--- | :--- |
| `my_list[c(1, 3)]` | **Cutting out** the 1st and 3rd sections of the pod. | A **new list** containing only the 1st and 3rd elements. |
| `my_list["B"]` | Cutting out the section labeled "B". | A **new list** containing just the element named "B". |

The output of this operation is still a list, which means it retains the structure and associated attributes of the original list.

```{r}
mylist[1]
str(mylist[1])
```

The double square bracket operator `[[]]` (or the dollar sign `$`) is like opening the pod and taking the pea out with your hand. `[[]]` allows you to access the contents of a single element, returning the pea itself (the actual data stored in that element).

```{r}
mylist[[1]]
str(mylist[[1]])
```

This means that the syntax to extract a specific value from a data structure stored in a list can be daunting. Below we extract the second column of a data frame stored at position 4 in the list **mylist**.  

```{r}
mylist[[4]][,2]
```

## Exercise

1. Create a list containing 1 character vector, a numeric vector, and a character matrix.

```{r,accordion=TRUE}
list.2 <- list(vec1 = c("hi", "ho", "merry", "christmas"),
               vec2 = 4:19,
               mat1 = matrix(as.character(100:81),nrow = 4))
list.2
```

2. Here is a data frame.

```{r}
dfr <- data.frame(letters, LETTERS, letters == LETTERS)
```

Add this data frame to the list created above.

```{r,accordion=TRUE}
list.2[[4]] <- dfr
```

3. Remove the the second entry in your list.

```{r,accordion=TRUE}
list.2[-2]
```

4. Create a new list that contains: a numeric vector, a character vector, and a logical vector.

```{r,accordion=TRUE}
list.a <- list(1:10, letters[1:5], c(T,F,T,F))
```

5. How long is your list, and how long are each of the elements in the list? Tip: remember you can apply a function on all elements of a matrix using the `apply` function. You can do the same for lists using `lapply`.

```{r,accordion=TRUE}
length(list.a)
lapply(list.a, FUN = "length")
```