Development

13 votes and 12 comments so far on Reddit

Welcome to The Quantum Well! - The Quantum Well - Powered by Obsidian Publish.

How to build a CLI for R, with npm.BackgroundThis blog post was triggered by a discussion on Twitter with MartinSkarzynski,who was looking for a way to build a CLI that launches an RScript.Here’s a way to do this using npm.Please note that this blog post won’t teach you how to build the commandline tool, it will quickly go over the way to create a system-widecommand line interface, using npm.If you want to learn more about building the utility, see thisfantastic series of blogpostsby Mark Sellor.Now, the idea is to have a CLI, i.e. a way to launch your utility with:$ mytoolAnd that, system-wide.What you’ll need An R script (script.R) with in it, for example:#!/usr/bin/env Rscript --vanillacli::cat_rule("yeay")cli::cat_bullet(Sys.time()) npm, which you can get fromthere.Let’s goCreate a new folder, and go inside it.mkdir cli && cd cliCreate the R Script there.echo '#!/usr/bin/env Rscript --vanilla' script.Recho 'cli::cat_rule("yeay")' script.Recho 'cli::cat_bullet(Sys.time())' script.RTry your script to see if it works:Rscript script.RNow launch an npm project:npm init -y(You can also run it without the -y to interactively add informationto the package.json.)Now the important part: add a "bin" value in the package.json, withthe name of the bin you want to create, and the path to the script,relatively to the package file. Here is an example of a package.json(I removed some elements).{ "name": "cli", "version": "1.0.0", "description": "CLI example with npm", "bin" : { "clir" : "./script.R" }, "author": "Colin Fay", "license": "MIT"}Install it globally (need sudo rights):sudo npm linkAnd, voilà! Open your terminal, and you’re done!clirOther way to go See the {littler}implementation

How to write a well-structured and understandable R program

Illustrating with a Principal Component Analysis

PnP PowerShell is an open source, community driven, PowerShell Module designed to work with Microsoft 365.

Comprehensive set of tools for scaffolding R interfaces to modules, classes, functions, and documentations written in other programming languages, such as Python.

Yet another post with a focus on package documentation! This time, we’ll cover vignettes a.k.a “long-form package documentation”, both basics around vignette building and infrastructure, and some tips for more maintainer- and user- friendliness. What is a vignette? Where does it live? In this section we shall go over basics of package vignettes. Vignette 101 In the “R packages” book by Hadley Wickham and Jenny Bryan, the vignettes chapter starts with “A vignette is a long-form guide to your package.

Statistics about Obsidian Plugins. New, trending, favorite Obsidian Plugins.

Turn your analyses into high quality documents, reports, presentations and dashboards with R Markdown. Use a productive notebook interface to weave together narrative text and code to produce elegantly formatted output. Use multiple languages including R, Python, and SQL. R Markdown supports a reproducible workflow for dozens of static and dynamic output formats including HTML, PDF, MS Word, Beamer, HTML5 slides, Tufte-style handouts, books, dashboards, shiny applications, scientific articles, websites, and more.

This post contains powershell commands to display progress status and show percentage of process completed for a running command or script and it will also display total amount time remains to complete current process.

Over 80K software development tech talks ranked algorithmically to surface the industry's best presentations from the past, present and future.

People. That is my leadership strategy – pure and simple. Find amazing people. Give them tools, frameworks, teammates, autonomy, and a…

Overview This walkthrough of the fundamentals of shell programming with Z shell (Zsh) and Bourne Again SHell (BASH) includes a comparison of similar components and features in R and RStudio. An alt…

This work is intended to foster best practices in reproducible data documentation and manipulation, statistical analysis, graphics, and reporting.

The project dedicated to empowering your PowerShell scripting.

Learn from a 10+ year PowerShell developer and a Microsoft PowerShell MVP on how to follow PowerShell best practices.

This is a post I’ve been meaning to write for some time now about {capsule}. {capsule} provides alternative workflows to {renv} for establishing and working with controlled package libraries in R. It also uses an renv.lock so it is compatible with {renv} - you can switch between doing things the {capsule} way and the {renv} or vice versa at any time. Introducing an R package I wrote nearly three years ago Carefully curating a controlled package environment the {renv} way can be kind of a chore.

Here's a tutorial on getting you started deploying your RShiny application on AWS Lightsail.

Designing an outstanding interface is not just about making it look nice with HTML and CSS. How do we handle interactivity, widget creation and data flow between R and JS? This is where JavaScript...

(2/4) Earning few JS skills for #RStats #shiny has loads of benefits:- Ease debugging (like {htmlwidgets} issues): https://t.co/Eitgrt8aG3.- Add new inputs/widgets: https://t.co/W9dR3IB65J- Optimize apps: less code from the server (R): https://t.co/ClqdAXok15 pic.twitter.com/IfQh4D477k— David Granjon (@divadnojnarg) May 30, 2022

[Disclaimer] The package presented in this blog post is stillexperimental at the time of writing these lines (2021-02-18), so itmight face some API changes in the future.Multi-page in {shiny}, and random thoughts about designing web applicationsSomething that has been bugging me for a while is the inability toreally share a specific part of a {shiny} dashboard, at leastnatively, using a specific path. In other words, I’ve always wanted tobe able to do something like my-uberapp.io/contact to share withsomeone a specific part of the app. And only this part.Another need I was facing, probably born out of building webapplications using NodeJS, is a natural way to manipulate endpoints,http requests and responses, so that I could do something prettycommon in web application: a home page, and next to it a login page thatverifies your identity, and redirects you to another page after settinga cookie. That of course, leads to the necessity of having the abilityto define various endpoints with a specific behavior for each: aunique UI, and a 18 server response. Pure, native, multi-endpointapplications, not one big ball of UI with parts hidden using JavaScript& CSS, and a global server function that might launch computation youdon’t need for your page.A few weeks back, I’ve decided to focus on this question. Multi-page in{shiny} is not new: I’ve found both{shiny.router}, and{blaze} that already implement aform of “multi-page”. Note: both these packages work perfectly, and they definitely do whatthey are designed for. There are the product of tremendous work withsmart implementations and my goal is in no way to undermine thesepackages. They just don’t answer the need I was having, hence my newapproach to this question. {brochure} will answer the needs for morelarge and complex applications built with {shiny}, while both the{shiny.router} & {blaze} should be easier to get started with.As far as I saw it, these two packages didn’t answer what I was lookingfor: “real” multi-page. Both {shiny.router} & {blaze} stillproduce a form of Single Page Application, but plays with the URL tomake you feel like you’re on a multi-page application.For example, {shiny.router} hides the parts of the app via JS & CSS –if you browse the source code, you’ll still see the hidden HTML. Thatalso implies that they are potential conflicts between ids: you have tobe sure that all your pages have unique id on their ouputs/inputs,otherwise the app doesn’t render, meaning that a “page” doesn’t haveits own server function. And of course, if you look at the Networktab of your browser developer tool, you’ll see that there is no newGET request made whenever a new “page” is loaded: this new page isstill the same application, with parts hidden through JavaScript.Hiding UI parts is a practice that has always been bothering me in termof performance – on a large {shinydashboard}, for example, you’ll betransferring the full HTML, CSS and JavaScript for the whole applicationwhen the app launches, even if the user never visits all the tabs.{shiny} is very smart when it comes to transferring CSS and JS files:for example, it will only serve the external resources for thesliderInput() if there is one in the app. But if you’ve got a{shinydashboard} dashboard with 19 pages, and the sliderInput() ison page 19 and will only be seen by 0.1% of the visitors, it will stillbe transferred to every user, regardless of whether or not they need it.This might result in lowering the performance of the app, as it might beraising the time to ‘First Meaningful Paint’, and of course it is awaste of resources, especially if your application is served to peoplewith low bandwidth, and/or browsing your app using cellular data. I knowthis is a question you’ve probably never asked yourself before ;) - butperformance is a pretty common question in the web development world,and something to be aware if you’re serving your app at scale. If thisis something you’re interested in, I suggest reading Why does speedmatter? from the Google dev center,and WebPerformancefrom Mozilla Web Docs. See also 14.2.2 Profiling{shiny}from the Engineering Shiny book.And, another final thing I wanted is a way to “flush” objects when youchange page: in the current implementation of {shinydashboard}, if youcreate something on tab1, it will still be there in tab2, takingsome space in the RAM, even if you don’t need it. It’s rather convenientbecause you don’t have to think about these things: you start a{shiny} session, and all the objects will still be there as long asthe session lives. But that also means that as soon as the sessionstops, there is no way to get the values back as they only live in theRAM, inside the session.On the other hand, with an implementation where every page gets its own{shiny} session, you need to find a way to identify the user with aform of id, save the relevant values (potentially write in a DB), thenfetch these values again using the id stored in the browser(typically, you have a session cookie in the browser that is used as akey to search the DB). A process forcing you to think more carefullyabout the data flow of your app. What that also means is that as theid is a cookie in the browser, whenever the app crashes, it will beable to reload to the exact same place as long as the cookie is still inthe browser (well, not exactly but you understand the idea).So, back to our first topic – what was I looking for? My goal was tofind a way to build {shiny} applications that are nativelymulti-page. In other words, an application that answers to a requeston /page2, not an app that silently redirect to /, nor anapplication that requires playing with /#!/page2 in the url.(Reminder, # is traditionally used as an anchor tag in a web page).And, of course, I wanted each page to have its own shiny session, itsown UI and server functions.I was also looking for a way to manipulate both the GET request fromthe server and the httpResponse that is sent back, for example inorder to set httpOnly cookies in the header of the HTTP Response orchange the HTTP status code, the same way you can manipulate thesewhen building an application with NodeJS for example. This is somethingthat you’d find in the{ambiorix} project by theamazing John Coene (who will get some SEOjuice thanks to this backlink). It’s an amazing approach that definitelyresonated with me as I’ve been a NodeJS & express.js user for sometime now, but that’s still far from the way you’d build things in{shiny}, and will probably require a lot of coding to get anapplication up and running (which is good, because I love coding, butlet’s use our {shiny} knowledge for now :) ).To sum up, I wanted a way to get closer to how you build applications inother languages, but will as little deviation as possible from the{shiny} way of building apps.Here comes {brochure}As said on the top of the article, this package is still a work inprogress, so you might see some changes in the near future :)[Disclaimer, again] The way you will build applications with{brochure} is different from the way you usually build {shiny} apps,as we no longer operate under the single page app paradigm. Please keepthis in mind and everything should be fine.InstallationYou can install the development version of {brochure} with:remotes::install_github("ColinFay/brochure")Minimal {brochure} Applibrary(shiny)library(brochure)## ## Attaching package: 'brochure'## The following object is masked from 'package:utils':## ## pagepage()A brochureApp is a series of pages that are defined by an href(the path/endpoint where the page is available), a UI and a serverfunction. This is conceptually important: each page has its own shinysession, its own UI, and its own server.brochureApp( # First page page( href = "/", ui = fluidPage( h1("This is my first page"), plotOutput("plot") ), server = function(input, output, session){ output$plot In your browser, you can now navigate to /, and to /page2.redirect()Redirections can be used to redirect from one endpoint to the other:brochureApp( page( href = "/", ui = tagList( h1("This is my first page") ) ), redirect( from = "/nothere", to = "/" ), redirect( from = "/colinfay", to = "https://colinfay.me" ))- In your browser, if you got to /nothere, and you’ll be redirectedto / .req_handlers & res_handlersSorry what?This is where things get more interesting.Each page, and the global app, have a req_handlers and res_handlersparameters, that can take a list of functions.An *_handler is a function that takes as parameter(s): For req_handlers, req, which is the request object (see belowfor when these objects are created). For example function(req){print(req$PATH_INFO); return(req)}. For res_handlers , res, the response object, & req , therequest object. For example function(res, req){ print(res$content);return(res)}. req_handlers must return req & res_handlers must returnres.They can be used to register log, or to modify the objects, or any kindof things you can think of. If you are familiar with express.js, youcan think of req_handlers as what express.js calls“middleware”. These functions are run when R is building the HTTPresponse to send to the browser (i.e, no server code has been run yet),following this process: R receives a GET request from the browser, creating a requestobject, called req The req_handlers are run using this req object, potentiallymodifying it R creates an httpResponse, using this req and how you definedthe UI The res_handlers are run on this httpResponse (first app levelres_handlers, then page level res_handlers), , potentiallymodifying it The httpResponse is sent back to the browserNote that if any req_handlers returns an httpResponse object, itwill be returned to the browser immediately, without any furthercomputation. This early httpResponse will not be passed to theres_handlers of the app or the page. This process can for example beused to send custom httpR...

Tiny things can separate life into “before” and “after”. Here is one. For almost a year I’ve been daily sending mental “thank you” to Ugo (@ugobas) who showed me how to re-organize panes in RStudio. Since then I’ve been spreading this tiny improvement so many times that I thought the tiny advise deserved a separate tiny post. Please note, below is an opinionated view of a comfortable UI improvement; feel free to ignore it if you don’t like. This advise is highly subjective, though, I really believe it is useful.

Key issues when adding persistent storage to a Shiny application, featuring {golem} app development and Digital Ocean serving

Function ExtractIcon { Param ( [Parameter(Mandatory=$true)] [string]$folder ) [System.Reflection.Assembly]::LoadWithPartialName('System.Drawing') | Out-N...

This document describes the AWS Well-Architected Data Analytics Lens, a collection of customer-proven best practices for designing well-architected analytics workloads. The Data Analytics Lens contains insights that AWS has gathered from real-world case studies, and helps you learn the key design elements of well-architected analytics workloads along with recommendations for improvement. The document is intended for IT architects, developers, and team members who build and operate analytics systems.

This post demonstrates how to extend the metadata contained in the Data Catalog with profiling information calculated with an Apache Spark application based on the Amazon Deequ library running on an EMR cluster. You can query the Data Catalog using the AWS CLI. You can also build a reporting system with Athena and Amazon QuickSight to query and visualize the data stored in Amazon S3.