Monthly Archives: January 2019

Amazon Web Services

How to Set Up Amazon S3 for Website Hosting

Leave a comment

Amazon Web Services S3 is a great way to host static websites. Here’s how to set up Amazon AWS S3 for Website Hosting. 

If you want to run WordPress on Amazon S3, see Serverless WordPress.

This tutorial assumes you’ve already got an Amazon Web Services account.

  1. Go to Services and search for or select S3.

    Select the S3 service
    Select the S3 service
  2. Click Create Bucket.

    Select create bucket
    Select create bucket
  3. Enter a name for your bucket, this must be unique. For instance bobs-cool-hosting.

    Give your bucket and a name and select a region
    Give your bucket and a name and select a region
  4. Select a region to host your bucket in. This will be geographically where your files are served from.  It’s best to choose a location close to where your users visit from. Click Next.
  5. The options can be left default, just click Next.

    Options can be left default
    Options can be left default
  6. Permissions set up is important. By default AWS S3 sets the bucket up to be secure and prevent it from being made publicly accessible. This is due to so many people just setting up buckets and accidentally or carelessly making them public, resulting security breaches. We want our bucket to be public because we’re hosting a website, so uncheck all the Public access settings and click Next.

    Permissions settings
    Permissions settings
  7. On the Review page you may be warned that this bucket may become public, that’s ok as we said so click Create bucket.

    Review page
    Review page
  8. So we’ve now created our bucket, as you can see here it’s marked “Objects can be public“. Click on the name of the bucket to open it.

    List of buckets
    List of buckets
  9. Click the Properties tab, then click Static website hosting.

    Select static website hosting
    Select static website hosting
  10. Click the option Use this bucket to host a website. Take note of the URL at the top, this will be used to access our website. Type in index.html as the index document and error.html as the error document. Click Save.

    Configure static website hosting
    Configure static website hosting
  11. If you now go to the URL we saw you’ll see it’s still saying 403 Forbidden. We now need to set up it’s permissions to enable public access.

    By default access is prevented
    By default access is prevented
  12. Click on the Permissions tab, then Bucket Policy. Copy in the following policy, being sure to change the bucket name in the Resource field from “my-serverless-wp” to match the name of your bucket. Click Save. 
    {
 "Version": "2012-10-17",
 "Statement": [
 {
 "Sid": "PublicReadGetObject",
 "Effect": "Allow",
 "Principal": "*",
 "Action": "s3:GetObject",
 "Resource": "arn:aws:s3:::my-serverless-wp/*"
 }
 ]
}

    Set up bucket policy
    Set up bucket policy
  13. Now create a test.html file with just a bit of text in it. On the Overview tab click Upload.

    Select Upload
    Select Upload
  14. Click Add File and Select the file you created and then click Next.
  15. Under Manage public permissions select Grant public read access to this object(s). Click Next.

    Set object to public
    Set object to public
  16. On the Set properties page the standard Storage Class is fine for this, click Next.

    Default properties are fine
    Default properties are fine
  17. Click Upload, our file will then be displayed in the list. 
  18. Go to the bucket url from step 10, enter this in a browser and add at the end “/test.html”. You should see your test.html page displayed.

    Test page is now displayed
    Test page is now displayed

Your S3 bucket is now ready to serve your website, but you’ll probably want to set up a DNS CNAME to give it a friendly domain name. I’ll explain how to do that in another article.

Amazon Web Services

Serverless WordPress (sort of)

Leave a comment

Here’s how I run my site(davidfindlay.com.au) in a sort of serverless way using Amazon S3. I say it’s sort of serverless because you still need an Apache/MySQL/WordPress installation, but it doesn’t need to be running all the time and can just run on your local computer.

Why host your WordPress site on Amazon S3?

Firstly S3 is very fast. WordPress hosted on LAMP has to bootstrap WordPress, talk to the database, process your request and generate a page before sending it to the browser. This all takes time. It makes sense if you host dynamic content. However if your content doesn’t change much, it doesn’t.

If you update your site maybe once a day, why have the HTML generated every time a visitor hits the site? With Static WordPress hosting you generate the HTML once when you make a change and the generated HTML is then served to each new visitor. This is much faster. 

As mentioned S3 is very fast, but it’s also scalable. If your site suddenly gets visited by 10000 people in an hour, S3 can handle it. Your typical WordPress installation on a LAMP hosting provider probably can’t. 

Secondly static hosting is more secure. Because your WordPress installation is hidden behind a firewall on your local network, you don’t have to worry about security updates and zero-day exploits as much. Sure you still should keep up to date, but because attackers don’t have any access to the PHP pages or database you’re kept much safer. Amazon has good security measures on S3 and as long as you use them, your S3 should be kept safe. 

Assumed Knowledge

  • Basic set up and installation of WordPress and WordPress plugins
  • Using hosting software such as MAMP or a LAMP server or Docker
  • Working at the command line

Step 1: WordPress installation

Firstly install WordPress locally. Perhaps using MAMP or on an Apache/MySQL/PHP installation on a linux box on your local network. How you do this part is up to you. I’ve actually got mine running on a small EC2 micro instance, that I just turn on and off when I want to make changes to my site.

No one will actually visit this WordPress installation, so it can just be local on your machine, not world accessible via the internet. Firewall it off so no one can reach it for maximum safety.

You’ll also need to install the AWS CLI. If you’re using an EC2 instance with an Amazon AMI, you’ll already have this. 

Step 2: Set up an S3 Bucket

You’ll need an Amazon Web Services account first, a free-tier account should be fine for most small sites for at least the first year. Afterwards you may need to pay, but S3 is really cheap.

There’s a lot of steps to setting up an S3 Bucket for web site hosting, so I’ve put them in a separate article here: How to Set Up Amazon S3 for Website Hosting.

Once you’ve got the S3 bucket set up return here.

Step 3: Install Simply Static WordPress Plugin

This is pretty much a standard WordPress plugin install, so I want explain it too much.

The Simply Static plugin automatically generates a plain html version of your site and exports it to a directory on your WordPress host. 

Static means that it’s plain HTML, no PHP. It can run on any sort of hosting without needing a PHP or MySQL installation. 

Once Simply Static is installed, activate it.

  1. Select Simply Static, then Settings from the left hand menu.
  2. Set Destination URLs to Use Relative URLs.

    Simply Static settings
  3. Set Delivery Method to Local Directory.

    Simply Static settings, continued
    Simply Static settings, continued
  4. Set Local Directory to a suitable location, for instance on my linux installation “/var/www/html_static”. Take note of this path as you may need to modify the script in Step 4 to match.

Step 4: Configure AWS IAMs user and AWS CLI

You’ll need an AWS IAM account set up to use the AWS CLI.

  1. Click Services at the top of the screen and in the search box type IAM. Click on the IAM option that appears in the drop down.
  2. Click Add User.

    Add IAM user
    Add IAM user
  3. Enter a user name such as “s3hosting”. Under Access Type, select Programmatic access. This is required so that the AWS CLI can use the user credentials. Click Next.

    Set up programmatic access
    Set up programmatic access
  4. Under Set Permissions, select Attach existing policies directly, then search for s3. Select the AmazonS3FullAccess policy. Click Next. Note that this policy means that using this AWS Access Key ID and Secret Key, someone could access any file in any bucket on your AWS account. This can be dangerous! 

    Select existing policy
    Select existing policy
  5. Continue through to the review page with default settings. The review page should look like this. Click Create User.

    Review and create user
    Review and create user
  6. You’ve now created the AWS CLI user. You’ll need the Access key ID and Secret access key displayed on this page for the next part of the process.

    Note access key id and secret key
    Note access key id and secret key

Next move back to your terminal where you’ve installed your WordPress.

Run the AWS Configure command. You’ll need to supply user IAM user Key ID and Secret Key as well as the default region, which should be the region that your S3 bucket is in:

aws configure
Configure AWS CLI
Configure AWS CLI

Create the following bash script and call it syncStatic.sh:

#!/bin/bash
aws s3 sync /var/www/html_static s3://my-serverless-wp/

Change ‘my-serverless-wp’ to match the name of your bucket and you may need to change ‘/var/www/html_static’ to match the local directory you set in Step 3.

Step 5: Generate Static HTML

In the WordPress Admin pages, select Simply Static from the side menu. Click Generate.

The log will show progress as the static html pages are generated. When the log shows “Done!” move to the next step.

Step 6: Sync to S3

In your terminal, run the syncStatic.sh script. It’ll quickly transfer the files to S3. If you’re running in a small EC2 instance this will be super quick, but a bit slower otherwise.

Step 7: Test the site

Go to your S3 public endpoint URL in your browser. For instance: http://my-serverless-wp.s3-website-ap-southeast-2.amazonaws.com/

You can get your URL from the S3 bucket configuration by going to Services->S3->Select your bucket->Properties->Static Web Hosting: 

Static website hosting url
Static website hosting url

After clicking on that URL or pasting it in your browser, you should be able to see your WordPress site and browse it.

Step 8: Set up DNS CNAME

Your site is now on the web, but it’s on an ugly Amazon AWS S3 url. You don’t want to direct people to that. 

The next step depends on how you want to host your site. You’ll need to set up CNAME(canonical name) which points your website domain to the the AWS S3 bucket address.

I’ll show how to do this for Amazon Route 53 DNS hosting in another article.

Programming

How to Serve an Angular Single Page Application using Django – Part 3

Leave a comment

Here’s part 1 of my series of how to serve an Angular 6 SPA web application in Django, without modifying the Angular CLI generated html. 

Putting it all in a Docker Container

I use Docker for development projects as it gives me a clean development environment where I can have all my dependancies isolated from other projects and stuff running on my system. 

Disclaimer: this may not be the best way to do Django and Angular in Docker. I chose this method because I wanted to have my Angular app served by Django to avoid CORS problems and just keep the architecture as close to what it’ll be in production as possible. Obviously it wouldn’t be running on Django’s runserver in production however. 

Here’s how to build a docker configuration to run my Django frontend and Angular backend together in the one container. 

First create a docker-compose.yml file:

version: '3'

services:
  db:
    image: postgres
  web:
    build:
      context: .
      dockerfile: Dockerfile
    working_dir: /code
    env_file:
      - web_variables.env
    command: sh devservers.sh
    volumes:
      - .:/code
    ports:
      - "8000:8000"
    depends_on:
      - db

This docker-compose file first creates a postgresql database container and then a web container using a Dockerfile. When the container is started it runs the devservers.sh script we created in Part 2.

It mounts the current directory(the django root) to /code within the container. It then exposes port 8000 inside the container to port 8000 outside the container. 

In the Dockerfile put:

FROM python:3
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
RUN curl -sL https://deb.nodesource.com/setup_10.x | bash -
RUN apt install nodejs
WORKDIR /code
ADD requirements.txt /code/
RUN pip install -r requirements.txt
ADD . /code/
RUN mkdir -p /code/static
WORKDIR /code/frontend
RUN npm install -g @angular/cli
RUN npm install
RUN ng build --outputPath=/code/static

This creates a new Docker container from the Python 3 official image. It creates a /code directory in the container.

It then installs node.js as as dependancy for the Angular CLI and installs all the Django project dependancies from the requirements.txt file. 

Finally to test that the environment is ready it copies in the code from the Django root, installs the Angular CLI globally, installs the Angular project dependancies and does a test build of them.

Note that in the docker-compose, we’re telling it to mount the current working directory as a volume on /code. So the /code path in the container will be replaced with the Django root from the host system. 

Really this means that the lines from ADD onwards in the Dockerfile are unnecessary. However I’ve left them in there to just ensure that that we know about it at image build time if the environment won’t be ready to use. 

Using the Docker environment

To start the dev servers run the following command in the Django project root:

docker-compose up -d

As your Django project root is mounted in the Docker container, any changes you make to the Django/Angular project will also be in /code in the Docker container. The Django manage.py runserver will automatically detect any changes on the Django side and recompile and serve them.

Angular build watch will detect changes to the Angular code in <Django project root>/frontend and rebuild them, putting the output SPA into /code/static where it’s served by Django’s static file serving. 

If you add new files or npm dependancies to the Angular project, you’ll need to restart the web container using:

docker-compose restart web

If you add new dependancies to the Django project do a full rebuild with:

docker-compose up --build -d 

In my case I actually run my dev install on a different machine than the one I write code on, so I just upload via Webstorm or PyCharm to the project root on my dev server. The changes are detected and ready to use within seconds. 

Uncategorized

How to Serve an Angular Single Page Application using Django – Part 2

Leave a comment

Here’s how to serve an Angular frontend and Django REST backend together, optionally in the same Docker container. 

Here’s part 2 of my series of how to serve an Angular 6 SPA web application in Django, without modifying the Angular CLI generated html. 

Running Django and Angular Auto-build Together

I’ve been working on a project that uses Angular on the frontend and Django REST Framework on the backend. Both Django and Angular have their own development servers that feature auto-recompile on change, which is really handy. 

I wanted to have the Angular app served by Django to avoid Cross Origin Request Validation(CORS) and because Django is handling uploaded files. There are other ways to deal with this in development but this is one way to do it.

The development project is structured in such a way that the frontend angular source root is a sub-directory of the Django project root. 

django_root/
    manage.py
    ... rest of django stuff ...
    frontend/
        angular.json
        ... rest of angular stuff ...
    static/

See my article on How to Serve an Angular SPA in Django for the details of how to configure Django to serve the Angular application

I created a shell script to start the two dev servers and called it devservers.sh:

#!/usr/bin/env bash

python3 manage.py makemigrations
python3 manage.py migrate
python3 manage.py runserver 0.0.0.0:8000 &

mkdir -p /code/static
cd frontend
npm install
ng build --watch --outputPath=/code/static/

Note: “/code/” may need to be changed to reflect the path to your django project root. In my case I’m serving this is Docker and installing the Django root in /code.

When the script runs it first makes the Django database migrations, loads them into the database server then runs it’s dev server on port 8000. It then makes sure the static files path has been created, installs any angular dependancies and build watches the angular code. 

Note instead of “ng run” I’m using “ng build –watch”. This watches the angular source code path like the ng run command and automatically builds source when it changes. However instead of serving it on port 4200, ng build deploys the compiled SPA to the outputPath, in this case /code/static/.

Put the devservers.sh script in the django root and run it. It’ll first start up Django and then Angular. Angular will take a minute or so to start up before you can access it. 

You’ll only need to stop and restart the devservers.sh script when new files(or dependancies) are added to the Angular app. This is because the ng build option only looks at changes in existing files, it doesn’t detect new files. 

In the next part I’ll show you how to put this all in a Docker container which is useful for keeping a clean development environment.