For a variety of reasons, I need to build a personal website (for myself). My preferred way of doing small website projects like this is to point a custom domain to Github Pages using Cloudflare… Free hosting!
My go-to method for designing sites has been to use a Ruby-based static site generator called Jekyll. Jekyll was developed by one of the founders of Github, so it’s baked right into Github Pages.
However, Node.js-based frameworks like React have become firmly entrenched in the industry now, so I thought it was high time that I try to actually build something I would actually use with React. Which led me to Gatsby.
I read up about Gatsby vs Next.js vs Jekyll before starting this project, but there isn’t anything up-to-date about the topic on the web.
I did manage to find StaticGen: a list of static site generators. When I counted there were 233 of them! That’s enough to give you complete analysis paralysis.
Build your statically-generated blog in Gatsby
I found an excellent step-by-step guide for building a blog in Gatsby, which is exactly what I was looking for.
I’ll check all of this into Github and Docker Hub eventually. I want to see if there is some sort of way to automatically detect when Gatsby has been updated and just automagically build and push a new image using Travis CI or something.
Not sure about Gatsby, but I know React and Node.js quite intimately. I’ve worked around each’s quirks and even built my own, more intuitive URL router and state manager (to hell with Redux and ReactRouter).
Regarding CD, we’re using Watchtower (a container updating containers!). It works quite seamlessly and magically to update all our deployed containers on our servers. Our CI/CD process is typically as follows:
Branch off the project’s development branch for the ticket you’re working on.
Write your code and commit as often as possible to your feature / bug branch.
Once you’re done, initiate a merge request to development.
Once the merge request is reviewed and accepted, let the CI/CD pipeline execute.
The CI/CD pipeline executes unit tests associated with the project.
When all tests pass, a container image is built for the project.
The container image is pushed to Docker Hub.
Watchtower then “watches” all deployed containers on the server and pulls updated images from the container registry (again, Docker Hub in our case).
It’s worth noting that we’re using Gitlab for our SCM and CI/CD. It really works a treat. Our whole project consists of a React-based SPA, a Node.js web gateway, 14 .NET Core microservices (REST API’s) each with their own MongoDB database and service workers performing automated jobs. We use NGINX as reverse proxy to the gateway and use Docker’s internal networking to access the microservices through the gateway.
We’re following strict Domain Driver Design (DDD) with a data sovereignty approach, so direct access between microservices is a big no-no. So for eventual consistency between microservices’ data repositories we’re using RabbitMQ for AMQP-based communication.
Everything deployed on the server is running in containers, with a volume mount for MongoDB to store the data on disk.
I am a designer, not a dev, that being said I need to have a fundamental understanding of how these frameworks behave and interact. So from a design perspective yes.
After some Googling and a lot of reading, what seems to be happening is that a program my project is looking for (cwebp), is not where it’s expecting it. Which is weird, because that binary is exactly at that location when I go and look for it.
The Glib-CRITICAL errors are apparently a red herring. They will happen if Node dies in the middle of of sharp processing the images.
The only recommendation I can find is “delete your node_modules and rebuild your project”.
Is this what developing on Node is like? “Oh! There’s a problem… delete node_modules and try again”?
Well, as a designer then I’d argue React is more applicable. Angular is very opinionated (i.e. it forces you to do things in a specific way, because that’s what a framework does), whereas React is a library that does the heavy DOM lifting for you - kind of in a way jQuery did all those years ago, except it forces you to not think about DOM at all. So with React you have more freedom to do things the way you want, but there are tools built in the ecosystem to help you get started.
The other modern one to throw in the mix is Vue, which is on the fence about whether it wants to be opinionated like Angular, or provide freedom like React. It’s newer than the other 2, so documentation isn’t as good as you’d find on them, especially considering its Chinese roots.
From personal experience I’d always recommend to go the React route. You can learn one powerful library that you can build powerful web applications as well as full native applications with. Angular and Vue have recent adaptations taking them to native app environments, but again Facebook has the edge of having React Native being the first battle tested JavaScript framework to build (compiled) native mobile apps.
No, that shouldn’t be necessary. node_modules is only the output folder for all the dependencies you pulled into the project. You’d usually just need to start up your app again. It’s the same with all self-hosted web applications though - once an unhandled exception is thrown, your app dies and you need to restart it. In the .NET Core world, the Kestrel web host works the same. It’s in essence a console app that listens for traffic on its configured ports. If an exception causes the app to die, it’s no longer listening on the ports and all web traffic will typically cause 502 gateway errors from your web proxy.
So a solution is to either handle those exceptions properly, or use a wrapper like forever or nodemon to automatically restart your app if it’s killed off by those nasty rogue exceptions. Luckily, both of those are just npm packages themselves and you only need to alter your app’s startup script to wrap it.
Did an npm rebuild, which was probably complete overkill because it ended up rebuilding some of the modules from source, including the one that was giving trouble.
Since I’m running a bare-bones Alpine image, I had to install things like make and gcc before the rebuild would work. Also had to install python, which was kind of weird. I’m probably making things hard on myself by using Alpine as my development image, but SSD space is precious.
I came upon npm rebuild when I went out to search for a way to the equivalent of a make clean in Node.js terms. Along the way I discovered gatsby clean, but also found that yarn doesn’t have rebuild unless you switch to the pre-release version of yarn 2.0.
This ultimately led me to an old-ish opinionated post about yarn vs npm, so I’ve decided to switch to using npm, even though the tutorial is written for yarn.
My project compiles now, though I’m no closer to understanding why it broke in the first place.
No shame in using Alpine as a development image - I use it as well! But I’m not even running the dev code on my local machine in a container… Just working from source. That’s the benefit of setting up a proper build pipeline - if your code compiles, it should work pretty much anywhere (given dependencies and whatnot).
So I write code, build, run and debug all locally via IDEs’ built-in tools and when I’m done, I initiate a merge request via Gitlab. The CI YAML file defines my CI process (using a couple of environment variables stored securely inside the Gitlab project configuration) and the pipeline the builds my Docker image using the Docker file.
Reason for the -win options for each script, is because *nix (and by extension Mac) and Windows treats paths differently… And we have people using both Windows laptops and Macbooks on this project.
I might do this… just inside Windows Subsystem for Linux. Which might just be exchanging the problems of one virtual environment for another, but all I’ve been doing for the past 2 hours is trying to fix a new self-inflicted problem and I suspect at least some of problems come from trying to run Gatsby in Alpine without understanding it properly.
So I got everything back working again, and I thought “Hey, let’s be fancy and make a little build script so that instead of typing npm run dev-docker every time I can just do an npm start.”
Because I came across this answer in Stack Overflow which talked about how to use if-env and I wanted to give it a shot.
“No problem,” I think. “It’s just as easy as typing npm install if-env --save.”
And then everything broke.
First, the npm install just hangs. My Docker container becomes completely inaccessible. I can’t docker exec into it, I can’t docker stop it, I can’t docker kill it. So I restart Docker.
Now nothing will work. When I run the gatsby develop develop command, it says the gatsby-cli module isn’t installed. When I try to run npm install, I get an error where it tries to move a file inside node_modules that doesn’t exist for some reason.
I’ve now deleted node_modules and then the re-install it died saying “npm ERR! Maximum call stack size exceeded”
So now I’m trying an empty node_modules with a clean cache: npm install --cache /tmp/empty-cache.
If that doesn’t work, the only thing I can think of is to completely redo my package.json file using npm install ... --save, because I had used yarn until now. Just in case there’s something weird in there that’s causing npm install to break.
Overall, my initial experience with React and Gatsby has not been pleasant.
I think the problem isn’t necessarily React and Gatsby, but the build pipeline and virtualisation architecture you’re trying to setup. If you just ran the Node.js app normally, serving from (what I assume would be) Express on a designated port, you would’ve been able to access it for development purposes easily in your browser.
I’m assuming you’re running Docker Desktop in Windows? You mentioned WSL, but seeing as WSL2 will only support a “proper” Linux kernel and it isn’t quite out yet, I’m going to assume you’re running in Windows. Then you have my sympathy… Docker Desktop in Windows is utterly, horrendously bad. I had it running in a production server from Hetzner with beefed up specs and it eventually crashed at least every hour when it hit some mid-level traffic. Since moving to Linux it hasn’t embarrassed me once.
Alpine is quite sparse yes, so it will take some time figuring out exactly which dependencies you need for your app, including OS-level packages that need to be installed (like wget, which isn’t in Alpine by default). I had to fine-tune one of my container images as well to install some dependencies, specifically for using Puppeteer (a Chromium wrapper that allows you to mimic browser actions using an API).
I’d urge you to stick with React and Node.js. It’s a wonderful toolset to get into modern web app development and the wealth of resources out there is staggering.
So after all that, it seems that the util-linux package I added to the Alpine image broke something in npm.
The Gatsby development server uses lscpu and gives a warning because it’s not available on Alpine by default, so I thought I’d be clever and include util-linux in my apk add directive. That seemed to have quietly broken npm.
UPDATE: Nope! Another basic test of npm on an empty project (npm init -y), breaks when I run npm install gatsby react react-dom --save.
For my next trick, I am running a basic benchmarks of yarn against npm, because regardless of what that earlier blog post said, npm feels slow.
I’ve added an exception to Windows Defender for my code directory, so we’ll see how much that speeds things up.
UPDATE:yarn is much faster with the exception to Windows Defender in place (based on one test). npm broke again, so it seems util-linux had nothing to do with it.
It looks like I’m going to have to use yarn for everything except if I need npm rebuild.
Phew! After many hours of tinkering, I am back in business and have worked all the way through the tutorial. I’ve even got my own fancy environment-aware package.json “scripts” section up an running using per-env.
I don’t really need any of the non-Docker stuff, but I wanted to see if I could make it work. (For reference: NODE_ENV in my Docker image is set to dev-docker.
Working through the tutorials means I’ve got some basic scaffolding for a website up and running and I know exactly how it fits together… Now I’m going to see if I can make it look half decent before populating it.
Yup… Because of how nomadic our lifestyle has become, I bought a Windows laptop with a GTX 1660 Ti so that I could spend some of my downtime on the road gaming.
I briefly had the option to upgrade to Windows 10 version 2004, which finally ships WSL2 as standard. But when I checked today to see if I could do the upgrade and hopefully get some better performance out of Docker, the Windows Updater said that I had to wait again.
If Microsoft says I should wait before downloading an update, I wait. A bunch of people who have installed 2004 have apparently had loads of trouble as a result.
Thanks! I toughed it out and seem to be back on track now. I’m curious to see how React handles styling. I’ve used Sass in the past and quite enjoyed it.
It won’t be any different to any other web development. React is only concerned with handling how markup is rendered and interacted with in the browser. The styling is still CSS at the end of the day, the same with any other way of getting the DOM updated in the browser (using static HTML, server-side rendering (SSR) or a single page app (SPA)). So you can use hand-coded CSS or preprocessors like SASS or LESS. I’ve used both SASS and LESS with React and you only need to install the necessary dependencies and configure them with a bundler / transpiler like Webpack.
You have the option of writing “JSS” in your JSX (the “markup” language of React), which is a fancy term for styling injected into the elements themselves. But this voids the decades old best practice of separation of concerns, so I stick to good old stylesheets (using SASS). There are advocates of both JSS and CSS in the React world, but at the end of the day you’re writing CSS rules that target elements on the DOM. I did investigate complex responsive layouts at some point using JSS, but many scenarios couldn’t intuitively be handled using it and the effort didn’t seem worth ditching CSS.
On that point… something else I’d like to check out at some stage is making the markup generated more semantic. One thing I really appreciated about Jekyll is that you can really hand-craft the HTML. The trade-off is that you don’t get that single page app feel that frameworks like React give you.
Sweet! I think Gatsby already uses Webpack (or at least something like it)… I’ll look into it.
This was what the tutorial I was following used (JSS), and your conclusion was my intuition as well.
A tremendous amount of work has also gone into frameworks like Bourbon, Foundation, and Bootstrap. I remember how impressed I was with Foundation the first time I tried it — great typography with an import.
Thank you, this is very good to know. I’ll definitely look into using a proper Sass framework.
