Methods

Collection of assessment data

Student performance on the first lecture exam in a 200-level Biology course was analyzed. The content assessed in all exams was biological diversity. However, the number and format of the questions used varied by semester. Individual student scores were collected using the new General Education Natural Sciences “scores” data workbook for 13 semesters. Student scores were automatically converted to a rubric score by the workbook using the equivalencies shown in Table 1.

Table 1: Conversion of percentages to rubric scores
Percent Correct Rubric Interpretation
0.0 to 49.0% 0 Unsatisfactory
50.0 to 59.9% 1 Beginning
60.0 to 69.9% 2 Developing
70.0 to 84.9% 3 Proficient
85.0 to 100.0% 4 Advanced

These workbook files contain personally identifiable information (PII) and are, therefore, subject to FERPA regulations. For this reason, they are not directly shared. Instead, they are permanently housed within the Proof_of_Concept folder under Core Competency: Natural Sciences in TracDat.

De-identification of student data

Copies of the 13 data files were downloaded from TracDat. An R aggregator script was used to read the data from these data sheets and concatenate it into one data set in a destructive process – the downloaded copies were deleted in the process. Student names and identification numbers were redacted and each student’s entry was given a unique eight-digit identifier - the Record.Key. These keys may be used for longitudinal studies in the future. The algorithm used is kept in an encrypted site and shared with no one. The de-identified data set contains 973 student entries and is formatted as a comma-delimited text file (BIOL200Data.csv).

Data provenance

Data provenance refers to a system that permits tracking of the origin, movement, modification, and utilization of data sets (Buneman, Khanna, and Wang-Chiew 2001). The provenance of General Education data will be explicitly declared to facilitate the reproducibility and extensibility of these studies.

Location of public website files

All files related to this report can be found online at the Open Science Framework (Nosek 2012). This site contains all of the files needed to reproduce this report from the de-identified data set. The site’s url is https://osf.io/t6u8m/.

Session information

This report was written using RStudio (RStudio Team 2015) and the R statistical programming language (R Core Team 2013). These products are free to download for PC, Macintosh, and Linux operating systems. The following information pertains to the session parameters used to generate this report. If you have trouble reproducing this report, it may be due to different session parameters. You may contact Dr. Franklund if you need assistance.

R version 3.4.1 (2017-06-30)

**Platform:** x86_64-apple-darwin15.6.0 (64-bit)

locale: en_US.UTF-8||en_US.UTF-8||en_US.UTF-8||C||en_US.UTF-8||en_US.UTF-8

attached base packages: grid, stats, graphics, grDevices, utils, datasets, methods and base

other attached packages: forestplot(v.1.7), checkmate(v.1.8.3), magrittr(v.1.5), dplyr(v.0.7.2), weights(v.0.85), mice(v.2.30), gdata(v.2.18.0), Hmisc(v.4.0-3), ggplot2(v.2.2.1), Formula(v.1.2-2), survival(v.2.41-3), lattice(v.0.20-35), moments(v.0.14), modeest(v.2.1) and pander(v.0.6.1)

loaded via a namespace (and not attached): gtools(v.3.5.0), splines(v.3.4.1), colorspace(v.1.3-2), htmltools(v.0.3.6), yaml(v.2.1.14), base64enc(v.0.1-3), rlang(v.0.1.2), foreign(v.0.8-69), glue(v.1.1.1), RColorBrewer(v.1.1-2), bindrcpp(v.0.2), plyr(v.1.8.4), bindr(v.0.1), stringr(v.1.2.0), munsell(v.0.4.3), gtable(v.0.2.0), htmlwidgets(v.0.9), evaluate(v.0.10.1), latticeExtra(v.0.6-28), knitr(v.1.17), highr(v.0.6), htmlTable(v.1.9), Rcpp(v.0.12.12), acepack(v.1.4.1), scales(v.0.5.0), backports(v.1.1.0), gridExtra(v.2.2.1), digest(v.0.6.12), stringi(v.1.1.5), bookdown(v.0.5), rprojroot(v.1.2), tools(v.3.4.1), lazyeval(v.0.2.0), tibble(v.1.3.4), cluster(v.2.0.6), pkgconfig(v.2.0.1), MASS(v.7.3-47), Matrix(v.1.2-11), data.table(v.1.10.4), assertthat(v.0.2.0), rmarkdown(v.1.6), rstudioapi(v.0.7), R6(v.2.2.2), rpart(v.4.1-11), nnet(v.7.3-12) and compiler(v.3.4.1)

Processing instructions

This project produced a computationally reproducible assessment report (this document). Anyone wishing to recreate this report from the source document will need to install the following on their computer:

  1. An installation of the R programming language
  2. An installation of the RStudio IDE
  3. An installation of LaTeX

The necessary source files include the de-identified data set (BIOL200Data.csv), Rmarkdown code files (index.Rmd, 01-Introduction.Rmd, 02-Methods.Rmd, 03-Results.Rmd, 04-Discussion.Rmd, and 05-References.Rmd), bibtex reference file (references.bib), and custom art file in the /art folder.

To process the files, you must first open the project in RStudio. Click on the “Build Book” button in the Build menu. Bookdown allows you to build this project as git_book (html site), pdf_book (via LaTeX), or epub_book (compatible with iBooks and other e-book readers).

Citation of this work

All of the de-identified data, analysis code, and documentation that constitute this report project may be freely used, modified, and shared. The de-identified data set, BIOL200Data.csv, is released under the Creative Commons CC0 license. All documentation, including README.md, Codebook.md, and this report, are released under the Creative Commons CC-BY licence. Any questions, comments, or suggestions may be sent to Dr. Franklund.