This tutorial details how to configure Flask to run on Docker along with Postgres, Nginx, and Gunicorn.

A guide to R development.

Simple mechanisms for defining and interpreting package options. Provides helpers for interpreting environment variables, global options, defining default values and more.

List all Ole DB Providers in Powershell

Modern JavaScript Tutorial: simple, but detailed explanations with examples and tasks, including: closures, document and events, object oriented programming and more.

Learn the best practices for designing web APIs that support platform independence and service evolution.

The Internet has become a critical part of almost every part of our society — it provides information, communication, and entertainment to billions of people every day, and enables coordination and collaboration between people and business across the globe. Unfortunately, this crucial piece of infrastructure is poorly understood by a large majority of the population; the myriad of technologies (both hard and soft) that let your laptop connect to your bank or to your mom’s Skype window could just as well be magic as far as average person in the street is concerned.

Here is how to create a progress bar in R that is versatile when working with loops, for example, by using package purrr.

All things R and RStudio

Data Modeling is a topic that never goes away. Sometimes I do reminisce about the good ol’ days of Kimball-style data models, it was so simple, straightforward, just the same thing for years. Then Big Data happened, Spark happened. Things just changed. There is a lot of new content coming out around Data Lakes and […]

Posts about clean-code written by Rubberduck VBA

300+ Free R programming books

When developing a package it is essential to track the changes you make to your code. This is especially vital if they are breaking changes which have implications for any code written that depends on your package, i.e. a major version bump. Although you can always look back at your version control history in git, it is also convenient to have documentation which summarises the changes. This is where the NEWS file comes in.

Welcome to The Quantum Well! - The Quantum Well - Powered by Obsidian Publish.

How to build a CLI for R, with npm.BackgroundThis blog post was triggered by a discussion on Twitter with MartinSkarzynski,who was looking for a way to build a CLI that launches an RScript.Here’s a way to do this using npm.Please note that this blog post won’t teach you how to build the commandline tool, it will quickly go over the way to create a system-widecommand line interface, using npm.If you want to learn more about building the utility, see thisfantastic series of blogpostsby Mark Sellor.Now, the idea is to have a CLI, i.e. a way to launch your utility with:$ mytoolAnd that, system-wide.What you’ll need An R script (script.R) with in it, for example:#!/usr/bin/env Rscript --vanillacli::cat_rule("yeay")cli::cat_bullet(Sys.time()) npm, which you can get fromthere.Let’s goCreate a new folder, and go inside it.mkdir cli && cd cliCreate the R Script there.echo '#!/usr/bin/env Rscript --vanilla' script.Recho 'cli::cat_rule("yeay")' script.Recho 'cli::cat_bullet(Sys.time())' script.RTry your script to see if it works:Rscript script.RNow launch an npm project:npm init -y(You can also run it without the -y to interactively add informationto the package.json.)Now the important part: add a "bin" value in the package.json, withthe name of the bin you want to create, and the path to the script,relatively to the package file. Here is an example of a package.json(I removed some elements).{ "name": "cli", "version": "1.0.0", "description": "CLI example with npm", "bin" : { "clir" : "./script.R" }, "author": "Colin Fay", "license": "MIT"}Install it globally (need sudo rights):sudo npm linkAnd, voilà! Open your terminal, and you’re done!clirOther way to go See the {littler}implementation

Statistics about Obsidian Plugins. New, trending, favorite Obsidian Plugins.

Over 80K software development tech talks ranked algorithmically to surface the industry's best presentations from the past, present and future.

People. That is my leadership strategy – pure and simple. Find amazing people. Give them tools, frameworks, teammates, autonomy, and a…

This work is intended to foster best practices in reproducible data documentation and manipulation, statistical analysis, graphics, and reporting.

[Disclaimer] The package presented in this blog post is stillexperimental at the time of writing these lines (2021-02-18), so itmight face some API changes in the future.Multi-page in {shiny}, and random thoughts about designing web applicationsSomething that has been bugging me for a while is the inability toreally share a specific part of a {shiny} dashboard, at leastnatively, using a specific path. In other words, I’ve always wanted tobe able to do something like my-uberapp.io/contact to share withsomeone a specific part of the app. And only this part.Another need I was facing, probably born out of building webapplications using NodeJS, is a natural way to manipulate endpoints,http requests and responses, so that I could do something prettycommon in web application: a home page, and next to it a login page thatverifies your identity, and redirects you to another page after settinga cookie. That of course, leads to the necessity of having the abilityto define various endpoints with a specific behavior for each: aunique UI, and a 18 server response. Pure, native, multi-endpointapplications, not one big ball of UI with parts hidden using JavaScript& CSS, and a global server function that might launch computation youdon’t need for your page.A few weeks back, I’ve decided to focus on this question. Multi-page in{shiny} is not new: I’ve found both{shiny.router}, and{blaze} that already implement aform of “multi-page”. Note: both these packages work perfectly, and they definitely do whatthey are designed for. There are the product of tremendous work withsmart implementations and my goal is in no way to undermine thesepackages. They just don’t answer the need I was having, hence my newapproach to this question. {brochure} will answer the needs for morelarge and complex applications built with {shiny}, while both the{shiny.router} & {blaze} should be easier to get started with.As far as I saw it, these two packages didn’t answer what I was lookingfor: “real” multi-page. Both {shiny.router} & {blaze} stillproduce a form of Single Page Application, but plays with the URL tomake you feel like you’re on a multi-page application.For example, {shiny.router} hides the parts of the app via JS & CSS –if you browse the source code, you’ll still see the hidden HTML. Thatalso implies that they are potential conflicts between ids: you have tobe sure that all your pages have unique id on their ouputs/inputs,otherwise the app doesn’t render, meaning that a “page” doesn’t haveits own server function. And of course, if you look at the Networktab of your browser developer tool, you’ll see that there is no newGET request made whenever a new “page” is loaded: this new page isstill the same application, with parts hidden through JavaScript.Hiding UI parts is a practice that has always been bothering me in termof performance – on a large {shinydashboard}, for example, you’ll betransferring the full HTML, CSS and JavaScript for the whole applicationwhen the app launches, even if the user never visits all the tabs.{shiny} is very smart when it comes to transferring CSS and JS files:for example, it will only serve the external resources for thesliderInput() if there is one in the app. But if you’ve got a{shinydashboard} dashboard with 19 pages, and the sliderInput() ison page 19 and will only be seen by 0.1% of the visitors, it will stillbe transferred to every user, regardless of whether or not they need it.This might result in lowering the performance of the app, as it might beraising the time to ‘First Meaningful Paint’, and of course it is awaste of resources, especially if your application is served to peoplewith low bandwidth, and/or browsing your app using cellular data. I knowthis is a question you’ve probably never asked yourself before ;) - butperformance is a pretty common question in the web development world,and something to be aware if you’re serving your app at scale. If thisis something you’re interested in, I suggest reading Why does speedmatter? from the Google dev center,and WebPerformancefrom Mozilla Web Docs. See also 14.2.2 Profiling{shiny}from the Engineering Shiny book.And, another final thing I wanted is a way to “flush” objects when youchange page: in the current implementation of {shinydashboard}, if youcreate something on tab1, it will still be there in tab2, takingsome space in the RAM, even if you don’t need it. It’s rather convenientbecause you don’t have to think about these things: you start a{shiny} session, and all the objects will still be there as long asthe session lives. But that also means that as soon as the sessionstops, there is no way to get the values back as they only live in theRAM, inside the session.On the other hand, with an implementation where every page gets its own{shiny} session, you need to find a way to identify the user with aform of id, save the relevant values (potentially write in a DB), thenfetch these values again using the id stored in the browser(typically, you have a session cookie in the browser that is used as akey to search the DB). A process forcing you to think more carefullyabout the data flow of your app. What that also means is that as theid is a cookie in the browser, whenever the app crashes, it will beable to reload to the exact same place as long as the cookie is still inthe browser (well, not exactly but you understand the idea).So, back to our first topic – what was I looking for? My goal was tofind a way to build {shiny} applications that are nativelymulti-page. In other words, an application that answers to a requeston /page2, not an app that silently redirect to /, nor anapplication that requires playing with /#!/page2 in the url.(Reminder, # is traditionally used as an anchor tag in a web page).And, of course, I wanted each page to have its own shiny session, itsown UI and server functions.I was also looking for a way to manipulate both the GET request fromthe server and the httpResponse that is sent back, for example inorder to set httpOnly cookies in the header of the HTTP Response orchange the HTTP status code, the same way you can manipulate thesewhen building an application with NodeJS for example. This is somethingthat you’d find in the{ambiorix} project by theamazing John Coene (who will get some SEOjuice thanks to this backlink). It’s an amazing approach that definitelyresonated with me as I’ve been a NodeJS & express.js user for sometime now, but that’s still far from the way you’d build things in{shiny}, and will probably require a lot of coding to get anapplication up and running (which is good, because I love coding, butlet’s use our {shiny} knowledge for now :) ).To sum up, I wanted a way to get closer to how you build applications inother languages, but will as little deviation as possible from the{shiny} way of building apps.Here comes {brochure}As said on the top of the article, this package is still a work inprogress, so you might see some changes in the near future :)[Disclaimer, again] The way you will build applications with{brochure} is different from the way you usually build {shiny} apps,as we no longer operate under the single page app paradigm. Please keepthis in mind and everything should be fine.InstallationYou can install the development version of {brochure} with:remotes::install_github("ColinFay/brochure")Minimal {brochure} Applibrary(shiny)library(brochure)## ## Attaching package: 'brochure'## The following object is masked from 'package:utils':## ## pagepage()A brochureApp is a series of pages that are defined by an href(the path/endpoint where the page is available), a UI and a serverfunction. This is conceptually important: each page has its own shinysession, its own UI, and its own server.brochureApp( # First page page( href = "/", ui = fluidPage( h1("This is my first page"), plotOutput("plot") ), server = function(input, output, session){ output$plot In your browser, you can now navigate to /, and to /page2.redirect()Redirections can be used to redirect from one endpoint to the other:brochureApp( page( href = "/", ui = tagList( h1("This is my first page") ) ), redirect( from = "/nothere", to = "/" ), redirect( from = "/colinfay", to = "https://colinfay.me" ))- In your browser, if you got to /nothere, and you’ll be redirectedto / .req_handlers & res_handlersSorry what?This is where things get more interesting.Each page, and the global app, have a req_handlers and res_handlersparameters, that can take a list of functions.An *_handler is a function that takes as parameter(s): For req_handlers, req, which is the request object (see belowfor when these objects are created). For example function(req){print(req$PATH_INFO); return(req)}. For res_handlers , res, the response object, & req , therequest object. For example function(res, req){ print(res$content);return(res)}. req_handlers must return req & res_handlers must returnres.They can be used to register log, or to modify the objects, or any kindof things you can think of. If you are familiar with express.js, youcan think of req_handlers as what express.js calls“middleware”. These functions are run when R is building the HTTPresponse to send to the browser (i.e, no server code has been run yet),following this process: R receives a GET request from the browser, creating a requestobject, called req The req_handlers are run using this req object, potentiallymodifying it R creates an httpResponse, using this req and how you definedthe UI The res_handlers are run on this httpResponse (first app levelres_handlers, then page level res_handlers), , potentiallymodifying it The httpResponse is sent back to the browserNote that if any req_handlers returns an httpResponse object, itwill be returned to the browser immediately, without any furthercomputation. This early httpResponse will not be passed to theres_handlers of the app or the page. This process can for example beused to send custom httpR...

Function ExtractIcon { Param ( [Parameter(Mandatory=$true)] [string]$folder ) [System.Reflection.Assembly]::LoadWithPartialName('System.Drawing') | Out-N...

This thoroughly revised guide demonstrates how the flexibility of the command line can help you become a more efficient and productive data scientist. You’ll learn how to combine small yet powerful command-line tools to quickly obtain, scrub, explore, and model your data. To get you started, author Jeroen Janssens provides a Docker image packed with over 100 Unix power tools—useful whether you work with Windows, macOS, or Linux.

Introduction

Obsidian Publish!

Top 20 Most Commonly Used Data Engineering Tools 2021

Evolution of Data Warehouse Architecture (Pic source: Databricks) With the evolution of scalable cloud technology and exponential growth of digital technologies, the preferred Data Warehouse (DW) a…

Programming is an interesting topic. There are so many different languages, frameworks and other tools that can be used to program. The world of programming

Find the best data modeling tools available on the market. What they do, and don't do.