UNCHARTED DATA: Interactive Tooltip Tables
How to include tables in your {ggiraph} tooltips.
UNCHARTED DATA: Introducing the {reactablefmtr} Package
An R package created to make the styling and customization of {reactable} tables easier.
UNCHARTED DATA: Using Crosstalk to Add User-Interactivity
Linking an interactive plot and table together with the crosstalk package.
Using Crosstalk to Add User-Interactivity
The goal is to link the reactable table I created to a plotly chart and provide additional filter options that control both the table and the chart.
An important note: in order to use crosstalk, you must create a shared dataset and call that dataset within both plotly and reactable. Otherwise, your dataset will not communicate and filter with eachother. The code to do this is SharedData$new(dataset).
If you expand the code below, you’ll see that the code to build a table in reactable is quite extensive. I will not go into the details in this post, but do recommend a couple great tutorials that I used to create the interactive table such as this tutorial from Greg Lin, and this from Tom Mock which really helped me understand how to use CSS and Google fonts to enhance the visual appeal of the table (see the “Additional CSS Used for Table” section below for more info).
If you have ever built something in Shiny before, you’ll notice that the crosstalk filters are very similar. You can add a filter to any existing column in the dataset. As you can see in the code below, I used a mixture of filter_checkbox and filter_select depending on how many unique options were available in the column you’re filtering. My rule of thumb is if there are more than five options to choose from it’s probably better to put them into a list in filter_select like I did with the Division filtering as to not take up too much space on the page.
For the layout of the data visualization, I used bscols to place the crosstalk filters side-by-side with the interactive plotly chart.
I then placed the reactable table underneath and added a legend to the table using tags from the htmltools package.
The final result is shown below. Feel free to click around and the filters and you will notice that both the plot and the table will filter accordingly. Another option is to drag and click on the plot and you will see the table underneath mimic the teams shown.
Design Patterns in R
Build robust and maintainable software with object-oriented design patterns in R. Design patterns abstract and present in neat, well-defined components and interfaces the experience of many software designers and architects over many years of solving similar problems. These are solutions that have withstood the test of time with respect to re-usability, flexibility, and maintainability. R6P provides abstract base classes with examples for a few known design patterns. The patterns were selected by their applicability to analytic projects in R. Using these patterns in R projects have proven effective in dealing with the complexity that data-driven applications possess.
R#r-development#design-patterns#r#software-architecture#object-oriented-programming#r-package#r-package-devt#api#r-api-devt
The most efficient way to manage snapshot tests in R.
Use CI and Github API
Snapshot testing gets difficult when there is more than one variant of the same result.
The reason why snapshot testing might be discouraging is due to the fact that snapshots will most likely fail due to environment settings. If one person runs the tests on a Mac and another on a Linux machine, the snapshots of rendered images will almost certainly be different. Comparing these snapshots will result in a failed test even though the code is correct.
Add CI to the mix, and you have a hot mess.
The easiest solution is to introduce variants.
Variants are versions of snapshots which were created on different environments.
In {testthat} variants are stored in separate directories. You can pass a name of the variant to the variant argument of testthat::test_snapshot. If you have a Linux, set variant = "linux", if you have a Mac, set variant = "mac".
Use snapshots generated on CI as the source of truth.
Don’t check in snapshots generated on your machine. Generate them on CI and download them to your machine instead.
Step 1: Archive snapshots on CI
Add this step to you CI testing workflow to allow downloading generated snapshots.
- name: Archive test snapshots if: always() uses: actions/upload-artifact@v3 with: name: test-snapshots path: | tests/testthat/_snaps/**/**/*
Step 2: Detect the environment to create variants
We can create a make_variant function to detect the version of the platform, as well as if we are running on CI.
This way even if we use the same OS on CI and locally, we can still differentiate between snapshots generated on CI and locally.
#' tests/testthat/setup.R is_ci <- function() { isTRUE(as.logical(Sys.getenv("CI"))) } make_variant <- function(platform = shinytest2::platform_variant()) { ci <- if (is_ci()) "ci" else NULL paste(c(ci, platform), collapse = "-") } # In tests: testthat::expect_snapshot(..., variant = make_variant())
Step 3: Ignore your local snapshots
Don’t check in snapshots generated on your machine. Add them to .gitignore instead.
Copy
tests/testthat/_snaps/linux-4.4
This way we can still generate snapshots locally to get fast feedback, but we’ll only keep a single source of truth checked in the repository.
Since you don’t track changes in local snapshots, you need to regenerate them before you start making changes to see if they change. It adds some complexity to the process, but it allows to keep the number of shared snapshots in the version control minimal.
Alternatively, you can keep local snapshots, but when doing code review, focus only on the ones generated on CI.
Step 4: Automate downloading snapshots from CI
To update snapshots generated on CI in Github, we need to:
Go to Actions.
Find our workflow run.
Download the test-snapshots artifact.
Unpack and overwrite the local snapshots.
testthat::snapshot_review() to review the changes.
Commit and push the changes.
This is a lot of steps. We can automate the most laborious ones with Github API.
The .download_ci_snaps function will:
Get the list of artifacts in the repository identified by repo and owner. It’ll search workflows generated from the branch we’re currently on. It will download the latest artifact with the provided name (in our case its “test-snapshots”) in the repository
Unzip them and overwrite the local copy of snapshots.
reactable: Column group definitions — colGroup
Use colGroup() to create column groups in a table.
Welcome and Getting Started – Next Generation Shiny Apps with {bslib}
Welcome to the workshop and hello, bslib!
Fast JSON, NDJSON and GeoJSON Parser and Generator
A fast JSON parser, generator and validator which converts JSON, NDJSON (Newline Delimited JSON) and GeoJSON (Geographic JSON) data to/from R objects. The standard R data types are supported (e.g. logical, numeric, integer) with configurable handling of NULL and NA values. Data frames, atomic vectors and lists are all supported as data containers translated to/from JSON. GeoJSON data is read in as simple features objects. This implementation wraps the yyjson C library which is available from .
Getting started with shinytest2
REST API in R with plumber
API and R Nowadays, it’s pretty much expected that software comes with an HTTP API interface. Every programming language out there offers a way to expose APIs or make GET/POST/PUT requests, including R. In this post, I’ll show you how to create an API using the plumber package. Plus, I’ll give you tips on how to make it more production ready - I’ll tackle scalability, statelessness, caching, and load balancing. You’ll even see how to consume your API with other tools like python, curl, and the R own httr package.
Nowadays, it’s pretty much expected that software comes with an HTTP API interface. Every programming language out there offers a way to expose APIs or make GET/POST/PUT requests, including R. In this post, I’ll show you how to create an API using the plumber package. Plus, I’ll give you tips on how to make it more production ready - I’ll tackle scalability, statelessness, caching, and load balancing. You’ll even see how to consume your API with other tools like python, curl, and the R own httr package
# When an API is started it might take some time to initialize # this function stops the main execution and wait until # plumber API is ready to take queries. wait_for_api <- function(log_path, timeout = 60, check_every = 1) { times <- timeout / check_every for(i in seq_len(times)) { Sys.sleep(check_every) if(any(grepl(readLines(log_path), pattern = "Running plumber API"))) { return(invisible()) } } stop("Waiting timed!") }
Oh, in some examples I am using redis. So, before you dive in, make sure to fire up a simple redis server. At the end of the script, I’ll be turning redis off, so you don’t want to be using it for anything else at the same time. I just want to remind you that this code isn’t meant to be run on a production server.
redis is launched in a background, , so you might want to wait a little bit to make sure it’s fully up and running before moving on.
wait_for_redis <- function(timeout = 60, check_every = 1) { times <- timeout / check_every for(i in seq_len(times)) { Sys.sleep(check_every) status <- suppressWarnings(system2("redis-cli", "PING", stdout = TRUE, stderr = TRUE) == "PONG") if(status) { return(invisible()) } } stop("Redis waiting timed!") }
First off, let’s talk about logging. I try to log as much as possible, especially in critical areas like database accesses, and interactions with other systems. This way, if there’s an issue in the future (and trust me, there will be), I should be able to diagnose the problem just by looking at the logs alone. Logging is like “print debugging” (putting print(“I am here”), print(“I am here 2”) everywhere), but done ahead of time. I always try to think about what information might be needed to make a correct diagnosis, so logging variable values is a must. The logger and glue packages are your best friends in that area.
Next, it might also be useful to add a unique request identifier ((I am doing that in setuuid filter)) to be able to track it across the whole pipeline (since a single request might be passed across many functions). You might also want to add some other identifiers, such as MACHINE_ID - your API might be deployed on many machines, so it could be helpful for diagnosing if the problem is associated with a specific instance or if it’s a global issue.
In general you shouldn’t worry too much about the size of the logs. Even if you generate ~10KB per request, it will take 100000 requests to generate 1GB. And for the plumber API, 100000 requests generated in a short time is A LOT. In such scenario you should look into other languages. And if you have that many requests, you probably have a budget for storing those logs:)
It might also be a good idea to setup some automatic system to monitor those logs (e.g. Amazon CloudWatch if you are on AWS). In my example I would definitely monitor Error when reading key from cache string. That would give me an indication of any ongoing problems with API cache.
Speaking of cache, you might use it to save a lot of resources. Caching is a very broad topic with many pitfalls (what to cache, stale cache, etc) so I won’t spend too much time on it, but you might want to read at least a little bit about it. In my example, I am using redis key-value store, which allows me to save the result for a given request, and if there is another requests that asks for the same data, I can read it from redis much faster.
Note that you could use memoise package to achieve similar thing using R only. However, redis might be useful when you are using multiple workers. Then, one cached request becomes available for all other R processes. But if you need to deploy just one process, memoise is fine, and it does not introduce another dependency - which is always a plus.
info <- function(req, ...) { do.call( log_info, c( list("MachineId: {MACHINE_ID}, ReqId: {req$request_id}"), list(...), .sep = ", " ), envir = parent.frame(1) ) }
#* Log some information about the incoming request #* https://www.rplumber.io/articles/routing-and-input.html - this is a must read! #* @filter setuuid function(req) { req$request_id <- UUIDgenerate(n = 1) plumber::forward() }
#* Log some information about the incoming request #* @filter logger function(req) { if(!grepl(req$PATH_INFO, pattern = "PATH_INFO")) { info( req, "REQUEST_METHOD: {req$REQUEST_METHOD}", "PATH_INFO: {req$PATH_INFO}", "HTTP_USER_AGENT: {req$HTTP_USER_AGENT}", "REMOTE_ADDR: {req$REMOTE_ADDR}" ) } plumber::forward() }
To run the API in background, one additional file is needed. Here I am creating it using a simple bash script.
library(plumber) library(optparse) library(uuid) library(logger) MACHINE_ID <- "MAIN_1" PORT_NUMBER <- 8761 log_level(logger::TRACE) pr("tmp/api_v1.R") %>% pr_run(port = PORT_NUMBER)
React
How to enhance your R shiny application with httpOnly Cookies
httpOnly Cookies are crucial for security, protecting against cross-site scripting attacks in R Shiny apps. Read more about them here.
R - Shiny#shiny#webdev#r-development#dev#r-shiny#shiny-server#security#cookies#http-only#cookie#cross-site scripting#xss#httpOnly
Draft for adding OAuth support to shiny by thohan88 · Pull Request #518 · r-lib/httr2
Info: This is a draft for discussion purposes. It's not a polished PR and currently includes minimal error handling and documentation. It may be big enough to warrant a separate package, bu...
Roxygen R6 Guide
mlr3: Machine Learning in R - next generation. Contribute to mlr-org/mlr3 development by creating an account on GitHub.
RMarkdown
Rstudio
The R Graph Gallery – Help and inspiration for R charts
The R graph gallery displays hundreds of charts made with R, always providing the reproducible code.
Create a table of shiny inputs
Create an interactive table of Shiny inputs by providing data and a table definition.
Handling Errors & Warnings in R | List of Typical Messages & How to Solve
How to fix error & warning messages in R - List of most common errors & warnings - Tutorials & examples on how to fix in RStudio
R - APIs
Shiny
Shiny is a package that makes it easy to create interactive web apps using R and Python.
Shiny was designed with an emphasis on distinct input and output components in the UI. Inputs send values from the client to the server, and when the server has values for the client to display, they are received and rendered by outputs.
You want the server to trigger logic on the client that doesn’t naturally relate to any single output.
You want the server to update a specific (custom) output on the client, but not by totally invalidating the output and replacing the value, just making a targeted modification.
You have some client JavaScript that isn’t related to any particular input, yet wants to trigger some behavior in R. For example, binding keyboard shortcuts on the web page to R functions on the server, or alerting R when the size of the browser window has changed.
agnostic, idiomatic data filter module for shiny
a small data filter module for shiny.
Rectangling
Rectangling is the art and craft of taking a deeply nested list (often
sourced from wild caught JSON or XML) and taming it into a tidy data set of
rows and columns. This vignette introduces you to the main rectangling tools
provided by tidyr: `unnest_longer()`, `unnest_wider()`, and `hoist()`.
How to Wrangle JSON Data in R with jsonlite, purr and dplyr - Robot Wealth
Working with modern APIs you will often have to wrangle with data in JSON format. This article presents some tools and recipes for working with JSON data with R in the tidyverse. We’ll use purrr::map functions to extract and transform our JSON data. And we’ll provide intuitive examples of the cross-overs and differences between purrr ... Read more
R - JSON Files
R - JSON Files - JSON file stores data as text in human-readable format. Json stands for JavaScript Object Notation. R can read JSON files using the rjson package.
How to read multilevel json data and convert to Data frame in R
this is exactly the data i have when i load data from mongolite no i need to remove the list with string concatenation
hendrikvanb
Working with complex, hierarchically nested JSON data in R can be a bit of a pain. In this post, I illustrate how you can convert JSON data into tidy tibbles with particular emphasis on what I’ve found to be a reasonably good, general approach for converting nested JSON into nested tibbles. I use three illustrative examples of increasing complexity to help highlight some pitfalls and build up the logic underlying the approach before applying it in the context of some real-world rock climbing competition data.