Discussion - Guide to deploy an Elixir/Phoenix app to AWS ECS

[joaquimadraz]

I'm opening this issue for a possible discussion of my blog post:
https://joaquimadraz.com/guide-to-deploy-an-elixir-phoenix-app-to-aws-ecs

[tim2CF]

Hello! Thanks for article.
I have some questions

1. What's a point to create Distillery releases when you have full-operational Erlang and Elixir inside Docker image (that is used to distribute program)?
2. How distributed Erlang works in case of using of AWS ECS? I mean it seems we need EPMD (Erlang Port Mapper Daemon) configuration and some other magical stuff that will create Erlang nodes cluster from our EC2 instances.
3. How you update your apps in runtime? Any examples of Erlang hot code reloading for EC2 instance?

[joaquimadraz]

Hey @tim2CF, thanks for the feedback.

The Dockerfile is split into two stages, build and release. The build stage uses Distillery to compile the application into Erlang binaries which also includes the Erlang Runtime (I forgot to mention this detail in the article: rel/config.exs#L37). Then release starts with plain Alpine Linux. Nothing from the build phase (meaning Erlang, Elixir, Phoenix, Mix, Node) is carried. There's no Erlang in there until we copy the compiled files into the release image: Dockerfile#L102.

In the end, the release image just has the Erlang runtime and our application binaries.

I still didn't dive into distributed Erlang on AWS. I totally going to use this project to dive into it.
Same with hot code reloading. I'll keep you posted.

[tim2CF]

Thanks!

[Merff]

@joaquimadraz Thanks for the guide!
I am currently on step docker run --rm -it -p 4002:4002 -e PORT=4002 subs:latest

docker build -t subs:latest \
  --build-arg PHOENIX_SECRET_KEY_BASE=super_secret_phoenix_key_base \ ...

This command works without any errors. But docker run --rm -it -p 4002:4002 -e PORT=4002 subs:latest have error:

Postgrex.Protocol (#PID<0.113.0>) failed to connect: ** (DBConnection.ConnectionError) tcp connect (localhost:5432): connection refused - :econnrefused
message=><<"connection not available because of disconnection">>

I am new in Docker, like that docker can not connect to the Postgres.
May be you know how to fix it?

[joaquimadraz]

Hey @Merff. Yes, what's probably happening is that docker is not able to connect to the host's Postgres.
Try using @docker.for.mac.host.internal instead of @localhost on the DATABASE_URL:

DATABASE_URL=postgresql://postgres:postgres@docker.for.mac.host.internal/subs_prod

Depending on your Docker version, you might need to use @docker.for.mac.host.internal instead of @ localhost for the database url so that the container can access the host’s Postgres. Read more about it here.

There's a note regarding that issue on the guide. I guess it's not properly placed :)

[Merff]

Hi @joaquimadraz. Thanks for the answer) But I don't use Mac, I use Ubuntu. And my project not an Umbrella, just standard Phoenix 1.3 project. How to correct change Dockerfile and may be other files? Thank you!)

[Merff]

Hi! I get "no basic auth credentials" while

docker push "$AWS_ECS_URL"/"$AWS_ECS_DOCKER_IMAGE"

Login works fine with

eval $(aws ecr get-login --no-include-email --region $AWS_DEFAULT_REGION)

[markdev]

Hi @joaquimadraz ,

I'm stuck on this step:

$ PORT=4001 _build/prod/rel/subs/bin/subs foreground
>>> Elixir.Subs.Tasks.ReleaseTasks.setup is either not defined or has a non-zero arity

I have module ReleaseTasks in apps/subs/priv/tasks/release_tasks.ex, under defmodule Elixir.Subs.Tasks.ReleaseTasks, with def setup do .... Any idea what could be causing this?

[joaquimadraz]

@Merff sorry for the late reply. Did you figure out what was the problem with the database connection?

Regarding the credentials problem, I'm not sure what can it be but I would say that it's a credentials problem. Make sure that the correct AWS keys are being used. Start by looking at ~/.aws/credentials default credentials.

[joaquimadraz]

@markdev Did you add the priv/tasks to the elixir compilation paths?
https://github.com/joaquimadraz/opensubs.io/blob/deployment/apps/subs/mix.exs#L32

[sean-clayton]

UPDATE

I got it all working!!

---

So I followed the steps and everything worked out in the end except... my app doesn't load. I'm completely new to AWS/any kind of devops stuff and am just an application developer.

Here is my project URL: https://gitlab.com/project-bourbon/mash

and here is my public url for the project: http://ec2-54-165-95-3.compute-1.amazonaws.com/

If there's anything glaringly obvious it'd be nice to know. I don't even know how to look at logs or anything :\

Here is my last deploy log as well: https://gitlab.com/project-bourbon/mash/-/jobs/62738365

Locally I can build production and provide the environment variables for my local db and it all works perfectly running from a docker container, so I know it's not that.

[joaquimadraz]

@sean-clayton I'm glad that you were able to do it!

I was checking you project's source code it has something to do with the connection with the database. Do you want to explain a bit the problem you faced?

Regarding the logs, what I do for now is accessing the EC2 instance, grab the docker container logs.

1. ssh to the EC2 instance
2. run docker ps to check the containers that are running

CONTAINER ID	IMAGE	COMMAND	...	PORTS
YourContainerID	subs_web:latest	subs_web/bin/subs...	...	tcp, 0.0.0.0:443->443 ...

3. Grab the container id (YourContainerID) and run docker logs YourContainerID

05:16:51.223 request_id=ar2f8mj2kg6se42lo5sabmgfl0cn6gi5 [info] Sent 200 in 168µs
05:59:35.608 request_id=mqgv4mqcvfgql16jgv9prnaptjce98e8 [info] GET /
05:59:35.608 request_id=mqgv4mqcvfgql16jgv9prnaptjce98e8 [info] Sent 200 in 173µs
06:35:06.920 request_id=5ge7814chchnn91qvvnpuhn75opulo8a [info] HEAD /
06:35:06.920 request_id=5ge7814chchnn91qvvnpuhn75opulo8a [info] Sent 200 in 144µs

This is not ideal but it's a good starting point to check if something went wrong on boot.

I'm working on logging everything to AWS CloudWatch. Would that be interesting for you?

[sean-clayton]

@joaquimadraz Yup, I dug a little bit and forgot that I had ssh access to my EC2 instance 😅 Quickly got access to logs after that. And about CloudWatch: That would definitely interest me. I'm kinda surprised there isn't a step on that in the wizard.

And about the problem I had... I'm not 100% sure why it started working. I think it was because I wasn't passing the DATABASE_URL var as an environment variable so Phoenix couldn't see it. Not 100% sure about that since I wasn't able to look at logs when that was occurring. I then passed down my DB settings in the docker-compose.yml file and it seemed to work after that. Normally, Phoenix would still start even if no DB connection could be made (meaning I could still see my routes and stuff, it would just fail on Repo things), but for me it wasn't even doing that. Then I remembered that I'm running migrations in the before script so I think that those kept failing and preventing Phoenix from even booting up, but these are all just hunches and suspicions.

My next steps that I'm going to try learning (as I said, completely new to this side of development):

• See if WebSockets work and if they don't, try to fix it
• Figure out how to SSL in AWS. Lots of acronyms now... (ACM, EC2, ECS, ELB, etc)
• Stick nginx in front of my ecs cluster to hold onto the certificate

Are these things "good practice" with AWS? Namely wondering about nginx there. I have no idea what the norm is on this field.

---

Update

Got an elastic load balancer with SSL support working :D

[blueangel0723]

@joaquimadraz I have the trouble to deploy the Elixir to AWS. Can you help me, please?