Learn from a 10+ year PowerShell developer and a Microsoft PowerShell MVP on how to follow PowerShell best practices.

This is a post I’ve been meaning to write for some time now about {capsule}. {capsule} provides alternative workflows to {renv} for establishing and working with controlled package libraries in R. It also uses an renv.lock so it is compatible with {renv} - you can switch between doing things the {capsule} way and the {renv} or vice versa at any time. Introducing an R package I wrote nearly three years ago Carefully curating a controlled package environment the {renv} way can be kind of a chore.

Here's a tutorial on getting you started deploying your RShiny application on AWS Lightsail.

Designing an outstanding interface is not just about making it look nice with HTML and CSS. How do we handle interactivity, widget creation and data flow between R and JS? This is where JavaScript...

(2/4) Earning few JS skills for #RStats #shiny has loads of benefits:- Ease debugging (like {htmlwidgets} issues): https://t.co/Eitgrt8aG3.- Add new inputs/widgets: https://t.co/W9dR3IB65J- Optimize apps: less code from the server (R): https://t.co/ClqdAXok15 pic.twitter.com/IfQh4D477k— David Granjon (@divadnojnarg) May 30, 2022

[Disclaimer] The package presented in this blog post is stillexperimental at the time of writing these lines (2021-02-18), so itmight face some API changes in the future.Multi-page in {shiny}, and random thoughts about designing web applicationsSomething that has been bugging me for a while is the inability toreally share a specific part of a {shiny} dashboard, at leastnatively, using a specific path. In other words, I’ve always wanted tobe able to do something like my-uberapp.io/contact to share withsomeone a specific part of the app. And only this part.Another need I was facing, probably born out of building webapplications using NodeJS, is a natural way to manipulate endpoints,http requests and responses, so that I could do something prettycommon in web application: a home page, and next to it a login page thatverifies your identity, and redirects you to another page after settinga cookie. That of course, leads to the necessity of having the abilityto define various endpoints with a specific behavior for each: aunique UI, and a 18 server response. Pure, native, multi-endpointapplications, not one big ball of UI with parts hidden using JavaScript& CSS, and a global server function that might launch computation youdon’t need for your page.A few weeks back, I’ve decided to focus on this question. Multi-page in{shiny} is not new: I’ve found both{shiny.router}, and{blaze} that already implement aform of “multi-page”. Note: both these packages work perfectly, and they definitely do whatthey are designed for. There are the product of tremendous work withsmart implementations and my goal is in no way to undermine thesepackages. They just don’t answer the need I was having, hence my newapproach to this question. {brochure} will answer the needs for morelarge and complex applications built with {shiny}, while both the{shiny.router} & {blaze} should be easier to get started with.As far as I saw it, these two packages didn’t answer what I was lookingfor: “real” multi-page. Both {shiny.router} & {blaze} stillproduce a form of Single Page Application, but plays with the URL tomake you feel like you’re on a multi-page application.For example, {shiny.router} hides the parts of the app via JS & CSS –if you browse the source code, you’ll still see the hidden HTML. Thatalso implies that they are potential conflicts between ids: you have tobe sure that all your pages have unique id on their ouputs/inputs,otherwise the app doesn’t render, meaning that a “page” doesn’t haveits own server function. And of course, if you look at the Networktab of your browser developer tool, you’ll see that there is no newGET request made whenever a new “page” is loaded: this new page isstill the same application, with parts hidden through JavaScript.Hiding UI parts is a practice that has always been bothering me in termof performance – on a large {shinydashboard}, for example, you’ll betransferring the full HTML, CSS and JavaScript for the whole applicationwhen the app launches, even if the user never visits all the tabs.{shiny} is very smart when it comes to transferring CSS and JS files:for example, it will only serve the external resources for thesliderInput() if there is one in the app. But if you’ve got a{shinydashboard} dashboard with 19 pages, and the sliderInput() ison page 19 and will only be seen by 0.1% of the visitors, it will stillbe transferred to every user, regardless of whether or not they need it.This might result in lowering the performance of the app, as it might beraising the time to ‘First Meaningful Paint’, and of course it is awaste of resources, especially if your application is served to peoplewith low bandwidth, and/or browsing your app using cellular data. I knowthis is a question you’ve probably never asked yourself before ;) - butperformance is a pretty common question in the web development world,and something to be aware if you’re serving your app at scale. If thisis something you’re interested in, I suggest reading Why does speedmatter? from the Google dev center,and WebPerformancefrom Mozilla Web Docs. See also 14.2.2 Profiling{shiny}from the Engineering Shiny book.And, another final thing I wanted is a way to “flush” objects when youchange page: in the current implementation of {shinydashboard}, if youcreate something on tab1, it will still be there in tab2, takingsome space in the RAM, even if you don’t need it. It’s rather convenientbecause you don’t have to think about these things: you start a{shiny} session, and all the objects will still be there as long asthe session lives. But that also means that as soon as the sessionstops, there is no way to get the values back as they only live in theRAM, inside the session.On the other hand, with an implementation where every page gets its own{shiny} session, you need to find a way to identify the user with aform of id, save the relevant values (potentially write in a DB), thenfetch these values again using the id stored in the browser(typically, you have a session cookie in the browser that is used as akey to search the DB). A process forcing you to think more carefullyabout the data flow of your app. What that also means is that as theid is a cookie in the browser, whenever the app crashes, it will beable to reload to the exact same place as long as the cookie is still inthe browser (well, not exactly but you understand the idea).So, back to our first topic – what was I looking for? My goal was tofind a way to build {shiny} applications that are nativelymulti-page. In other words, an application that answers to a requeston /page2, not an app that silently redirect to /, nor anapplication that requires playing with /#!/page2 in the url.(Reminder, # is traditionally used as an anchor tag in a web page).And, of course, I wanted each page to have its own shiny session, itsown UI and server functions.I was also looking for a way to manipulate both the GET request fromthe server and the httpResponse that is sent back, for example inorder to set httpOnly cookies in the header of the HTTP Response orchange the HTTP status code, the same way you can manipulate thesewhen building an application with NodeJS for example. This is somethingthat you’d find in the{ambiorix} project by theamazing John Coene (who will get some SEOjuice thanks to this backlink). It’s an amazing approach that definitelyresonated with me as I’ve been a NodeJS & express.js user for sometime now, but that’s still far from the way you’d build things in{shiny}, and will probably require a lot of coding to get anapplication up and running (which is good, because I love coding, butlet’s use our {shiny} knowledge for now :) ).To sum up, I wanted a way to get closer to how you build applications inother languages, but will as little deviation as possible from the{shiny} way of building apps.Here comes {brochure}As said on the top of the article, this package is still a work inprogress, so you might see some changes in the near future :)[Disclaimer, again] The way you will build applications with{brochure} is different from the way you usually build {shiny} apps,as we no longer operate under the single page app paradigm. Please keepthis in mind and everything should be fine.InstallationYou can install the development version of {brochure} with:remotes::install_github("ColinFay/brochure")Minimal {brochure} Applibrary(shiny)library(brochure)## ## Attaching package: 'brochure'## The following object is masked from 'package:utils':## ## pagepage()A brochureApp is a series of pages that are defined by an href(the path/endpoint where the page is available), a UI and a serverfunction. This is conceptually important: each page has its own shinysession, its own UI, and its own server.brochureApp( # First page page( href = "/", ui = fluidPage( h1("This is my first page"), plotOutput("plot") ), server = function(input, output, session){ output$plot In your browser, you can now navigate to /, and to /page2.redirect()Redirections can be used to redirect from one endpoint to the other:brochureApp( page( href = "/", ui = tagList( h1("This is my first page") ) ), redirect( from = "/nothere", to = "/" ), redirect( from = "/colinfay", to = "https://colinfay.me" ))- In your browser, if you got to /nothere, and you’ll be redirectedto / .req_handlers & res_handlersSorry what?This is where things get more interesting.Each page, and the global app, have a req_handlers and res_handlersparameters, that can take a list of functions.An *_handler is a function that takes as parameter(s): For req_handlers, req, which is the request object (see belowfor when these objects are created). For example function(req){print(req$PATH_INFO); return(req)}. For res_handlers , res, the response object, & req , therequest object. For example function(res, req){ print(res$content);return(res)}. req_handlers must return req & res_handlers must returnres.They can be used to register log, or to modify the objects, or any kindof things you can think of. If you are familiar with express.js, youcan think of req_handlers as what express.js calls“middleware”. These functions are run when R is building the HTTPresponse to send to the browser (i.e, no server code has been run yet),following this process: R receives a GET request from the browser, creating a requestobject, called req The req_handlers are run using this req object, potentiallymodifying it R creates an httpResponse, using this req and how you definedthe UI The res_handlers are run on this httpResponse (first app levelres_handlers, then page level res_handlers), , potentiallymodifying it The httpResponse is sent back to the browserNote that if any req_handlers returns an httpResponse object, itwill be returned to the browser immediately, without any furthercomputation. This early httpResponse will not be passed to theres_handlers of the app or the page. This process can for example beused to send custom httpR...

Tiny things can separate life into “before” and “after”. Here is one. For almost a year I’ve been daily sending mental “thank you” to Ugo (@ugobas) who showed me how to re-organize panes in RStudio. Since then I’ve been spreading this tiny improvement so many times that I thought the tiny advise deserved a separate tiny post. Please note, below is an opinionated view of a comfortable UI improvement; feel free to ignore it if you don’t like. This advise is highly subjective, though, I really believe it is useful.

Key issues when adding persistent storage to a Shiny application, featuring {golem} app development and Digital Ocean serving

Function ExtractIcon { Param ( [Parameter(Mandatory=$true)] [string]$folder ) [System.Reflection.Assembly]::LoadWithPartialName('System.Drawing') | Out-N...

This document describes the AWS Well-Architected Data Analytics Lens, a collection of customer-proven best practices for designing well-architected analytics workloads. The Data Analytics Lens contains insights that AWS has gathered from real-world case studies, and helps you learn the key design elements of well-architected analytics workloads along with recommendations for improvement. The document is intended for IT architects, developers, and team members who build and operate analytics systems.

This post demonstrates how to extend the metadata contained in the Data Catalog with profiling information calculated with an Apache Spark application based on the Amazon Deequ library running on an EMR cluster. You can query the Data Catalog using the AWS CLI. You can also build a reporting system with Athena and Amazon QuickSight to query and visualize the data stored in Amazon S3.

Thoughts on packaging python libraries — Note This is about packaging libraries, not applications. ⸻ All the advice here is implemented in a project template (with full support for C...

This is the unofficial documentation for Obsidian plugin development.

Obsidian: A knowledge base that works on local Markdown files.

If the original data is spread out in a number of files, we can read them in parallel (on multi-core) machines to save time. Ideally. However, it seems a bit tricky with python.

Python Flask tutorial showing IntelliSense, debugging, and code navigation support in Visual Studio Code, the best Python IDE.

Obsidian is an amazing tool for developer and developer advocates. You can use it as a second brain, for content creation, and more.

This thoroughly revised guide demonstrates how the flexibility of the command line can help you become a more efficient and productive data scientist. You’ll learn how to combine small yet powerful command-line tools to quickly obtain, scrub, explore, and model your data. To get you started, author Jeroen Janssens provides a Docker image packed with over 100 Unix power tools—useful whether you work with Windows, macOS, or Linux.

This tip is focused on the concepts of using Git Workflow with SQL Database Projects.

I was tinkering with the idea of finding the right way to help others to identify the resources that give you bang for the buck when it comes to upskilling yourself in data engineering… and this is what I came up with. The best data engineering courses, learning materials, data engineering training.

Introduction

Many organizations are benefiting from the performance gains and ease of administration of the Amazon Aurora database platform. The idea of breaking free from commercial licenses, backup administration, and data center maintenance is always a welcome thought. But where to begin, and how complex is the journey to migrate from SQL Server to Amazon Aurora […]

This article reviews the differences between stored procedures and functions in Postgres and the types of functionality they provide.

In this article, we explain the concept of Stored Procedures, and how to write them effectively in PostgreSQL 9.6. PostgreSQL is an open source database

The book that teaches SQL to developers: Learn to replace thousands of lines of code with simple queries!