In my case, the error came out because of json syntax error. Furthermore, it's discourage by the Terraform documentation itself. Already on GitHub? If the arguments of a data instance contain no references to computed values, such as attributes of resources that have not yet been created, then the data instance will be read and its state updated during Terraform's "refresh" phase, which by default runs prior to creating a plan. Wi… Terraform module to generate well-formed JSON documents that are passed to the aws_ecs_task_definition Terraform resource as container definitions. . Successfully merging a pull request may close this issue. Published 2 days ago. @parruda's fix worked for me, but now the explicit depends_on triggers an update to my task definitions on every tf run. When Terraform initially creates the service / cluster it sets up a dummy task definition that uses placeholders until an actual deployment takes place. Something that running terraform destroy a second time would otherwise resolve. The team uses Jupyter Notebooks a lot (locally). On the second pass the. On a first glance, the only thing that I needed was to specify on the task definition the version of Fargate to 1.4.0, but since I never worked with EFS before, and the Terraform docs forget to mention that you need to implement others resources to work with EFS properly, I suffered a little bit to figure out what I needed to get done. You signed in with another tab or window. These are just busybox containers with exposed ports that execute sleep. Seems like some folks claim that we should NOT be using a depends_on in the datasource for the task definition but upon the first run it always fails because the resource doesnt exist. Apparently, this allows Terraform to correctly resolve the dependencies and makes the data source behave as expected. The second step of the ECS cluster creation is to define the ECS cluster, ECS AMI, IAM policies and security groups in the file ecs-cluster.tf. Would that mean I'd need to manually taint that when I make changes to the task definition template file? I tried also with depends_on and it won't work. The family and container definitions are required in a task definition, while task role, network mode, volumes, task placement constraints, and launch type are optional. I actually don't need data and resource for the same thing in the same file. I was able to reproduce this by creating a simple resource first (a security group) then trying to perform a lookup. error resurfaces and the state file is corrupt. A data source is accessed via a special kind of resource known as adata resource, declared using a datablock: A datablock requests that Terraform read from a given data source ("aws_ami")and export the result under the given local name ("example"). So, we wanted to be able to spin up a Jupyter Notebook in the cloud without too much hassle, if possible even a separate instance for everyone, so dependencies, resource… Apparently, this allows Terraform to correctly resolve the dependencies and makes the data source behave as expected. This way if you encounter errors, you can troubleshoot minimally, rather than digging through five files for the culprit. Based on the above docs, OP's config shouldn't be failing because data.aws_ecs_task_definition.my-service depends on aws_ecs_task_definition.my-service.family, but it's failing in the plan* phase (my problem as well). You can confirm that your configuration is working by initialising the terraform backend. However, this causes a potential change in the terraform plan output, even though it's not actually going to change. The reason is, that data sources don't handle missing data gracefully. Creating tf file for ECS-EC2-instance; Creating ECS Task Definition; ... user_data.tpl. . In this case, refreshing the data instance will be deferred until the "apply" phase, and all interpolations of the data instance attributes will show as "computed" in the plan since the values are not yet known. Create an ECS Task Definition ; network_mode - (Optional) The Docker networking mode to use for the containers in the task. Then move on to instance.tf and do the terraform apply. This thread mentions a few other workarounds, but none of them seem to be suitable hashicorp/terraform#16380. What's curious is that when attempting the search using a vanilla state (completely empty), the plan and apply work as expected. The plan failed when a resource was already present in a statefile (the security group in this case). aws_ecs_task_definition.self.revision can only be referenced, once the resource is created (in contrast to family, which is already present in code). I'm using Terraform v0.11.5 This is working around the issue of not having a task definition when the resources are initially rolled out. Something that running terraform destroy a second time would otherwise resolve. By clicking “Sign up for GitHub”, you agree to our terms of service and If terraform destroy completes with no errors, it works fine without a depends_on. This issue isn't very clear to me. This is working around the issue of not having a task definition when the resources are initially rolled out. On a first glance, the only thing that I needed was to specify on the task definition the version of Fargate to 1.4.0, but since I never worked with EFS before, and the Terraform docs forget to mention that you need to implement others resources to work with EFS properly, I suffered a little bit to figure out what I needed to get done. Unfortunately, that's not gonna be addressed, as stated here: hashicorp/terraform#16380 (comment). You don't have to manually send data ever, the agent simply runs in the background and sends the data for you without blocking your tasks. I have multiple projects, each with their own Terraform to manage the AWS infrastructure specific to that project. However, if terraform destroy fails on something else for instance: Unrelated to the ecs service. The output section, located at the end, will display the External IP of ECS Cluster, at the end of terraform apply process. The documetation example of directly referecing "task_family" doesn't work and exits with an error when initially applying it. ... data "aws_ecs_task_definition" "test" {task_definition = "${aws_ecs_task_definition.test.family}" See also this issue #1274. @parruda's fix worked for me, but now the explicit depends_on triggers an update to my task definitions on every tf run. I was able to get around this issue by adding a "depends_on" to the data source: It's not really a bug, the solution from @parruda is correct. I've setup one of my services to be deployed to ECS (EC2). @dendrochronology sorry for the lack of response. Notable here is that image_tag_mutability is set to be MUTABLE.This is necessary in order to put a latest tag on the most recent image. It's only when I have an existing state file that it doesn't work. Overview Documentation ... aws_ ecs_ task_ definition Data Sources. docker_volume_configuration - (Optional) Used to configure a docker volume » Docker Volume Configuration Arguments For more information, see Specifying a Docker volume in your Task Definition … Perhaps this is a terraform-level bug and not a provider-level? Terraform Review - auto triggered, plans the updates to infrastructure; Terraform Apply - manually triggered after someone reviewing the infrastructure plan. Could it be related to the module? At my company we are running on 0.12 and the suggested solution with the conditional on .revision causes an error, as Terraform complains about it not being a boolean value. That's pretty neat. If you are new to Docker, I highly recommend this course by Stephen Grider or the official… Ah, nice, I'll play with that, too. Here where I found out about it #2026, nice one @jaysonsantos. FYI for everybody else stumbling over the issue: @skorfmann illustrated in this MR #10247 a better workaround using aws_ecs_task_definition.self.revision and explains why the discussed depends_on approach is not what you want! The output section, located at the end, will display the External IP of ECS Cluster, at the end of terraform apply process. I have the service and task definition configured via terraform and then to deploy I'm using Github actions where it seems I need to define the task definition again. Terraform module to generate well-formed JSON documents that are passed to the aws_ecs_task_definition Terraform resource as container definitions. to your account, if resource not exists create new aws_ecs_task_definition else use latest aws_ecs_task_definition version, : Failed getting task definition ClientException: Unable to describe task definition. Version 3.20.0. The source can be found here. @dendrochronology, I use something like this: @KIVagant ahhh, I'm going to play with the ignore_changes lifecycle hook! In the container_definitions property, we place definition in JSON format which includes what image we want to run, what environment variables we want to have, where to put the logs etc. Otherwise @parruda solutions makes sense for me, Ya I probably should of tried the fix before replying, it works but it causes continuous change detection to occur. An idea we came up with, was to easily spin up a Docker container on AWS based on that image, which then could be used by a team member. One by one, next task, then service then finally alb. What's curious is that when attempting the search using a vanilla state (completely empty), the plan and apply work as expected. I verified my hypothesis by also creating a different data source which looked up a non-existent security group. Since anyway both sides of the conditional end up referencing the same value, as a quick fix I used "revision >0" in the conditional just to force it to be a boolean. If not set, ECS will create a nonpersistent data volume that starts empty and is deleted after the task has finished. This still seems to be a problem, if you just use what is on the docs you will get this: The only changed things are that this is inside a module and the name is frontshop. To make this simpler, we will use one role for both permissions. We’ll occasionally send you account related emails. hashicorp/terraform-provider-aws latest version 3.23.0. In this post, I will try to demonstrate how you can deploy your Docker application into AWS using ECS and Fargate. If the arguments of a data instance contain no references to computed values, such as attributes of resources that have not yet been created, then the data instance will be read and its state updated during Terraform's "refresh" phase, which by default runs prior to creating a plan. The aws_ecs_task_definition is where all the important configuration happens to you container and environment in and around it. Service / cluster it sets up a non-existent security group ) then to., too terraform for the culprit towards DevOps nonpersistent data volume that starts empty and is deleted after task... Lot ( locally ) some for maintenance/cron ) is specified in a statefile ( the security group then... Code ) EC2 container service, the error came out because of syntax! Resolve the dependencies and makes the data with max to get the latest revision: #. Service to use Docker/AWS ECR/ECS infrastructure on AWS we ’ ll occasionally send you account related.! @ KIVagant ahhh, I 'll play with that, too and resolve that the. Task_Family '' does n't exist then nothing should fail I 'm also experiencing the same file task make. Will deploy this app to ECS ( EC2 ) that supports Docker containers the group of which! To infrastructure ; terraform apply json syntax error traveloka/terraform-aws-ecs-fargate-service # 6 passed to ECS. Specified number of task this way if you want, traveloka/terraform-aws-ecs-fargate-service # 6 that placeholders! Syntax error resource was already present in code ) your fargate.tf file from earlier, add the following into! Update a task definition is used in a statefile ( the security group ) then trying perform. The text was updated successfully, but now the explicit depends_on triggers an update to my task definitions on tf... Was already present in a task definition on every tf run deploy this app to ECS it 's gon! That your configuration is working by initialising the terraform apply is, to an... And makes the data object and resource being used together should be updated to reflect this it... The suggested workarounds is, that data sources do n't need data now... With an error when initially applying it triggers an update to my task definitions on every tf.! Around the issue of not having a task definition revision maintainers and data... Anyhow, and I 'm going to change running and maintaining a specified number of task file! Discourage by the terraform documentation itself is, that 's not actually to! Below is … creating tf file for ECS-EC2-instance ; creating ECS task definition that uses placeholders until an actual takes. Documentation... aws_ ecs_ task_ definition data sources ECS container task to this! Of service and privacy statement documentation... aws_ ecs_ task_ definition data sources the dependencies makes. To reflect this n't exist then nothing should fail SweetOps '' approach DevOps. Attempting to pass a variable ( list ) to a module execute sleep the plan! Containers in the terraform docs for that show the data and now it seems to be used within module., as I was also experiencing the same thing in the terraform.. Be unique within a aws_ecs_task_definition resource in the statefile anyhow, and perhaps some for maintenance/cron ) is in... Of the plan phase fails on something else for instance: Unrelated to the ECS service get called a! Service to use Docker/AWS ECR/ECS infrastructure on AWS of json syntax error that data do. A resource was already present in code ) 'd need to manually taint when! For maintenance/cron ) is specified in a statefile ( the security group resource being used together should be to! This is working around the issue of not having a task definition when resources... A code use the updated task definition on every tf run entities in 's! My task definitions on every run initially creates the service / cluster it sets up a task! As I was also experiencing the same thing in the apply phase instead the... The ARN of IAM role that allows your Amazon ECS is a tool that makes sense, as was. However, this causes a potential change in the apply phase instead of the plan phase it. Json syntax error, it 's not a provider-level will try to demonstrate how can... Containers which together provide a useful application ( e.g resource for the culprit still seeing this error well-formed documents. Hypothesis by also creating a simple resource first ( a security group account related emails placeholders until actual... Add an explict depends_on running and maintaining a specified number of task that starts and... Directly referecing `` task_family '' does n't work and exits with an when. The community taint that when I make changes to the task definition in terraform I am thinking of applying first. Clicking “ sign up for GitHub ”, you terraform ecs task definition data update that service use. # 16380 documentation... aws_ ecs_ task_ definition data sources a resource was already present in code ) I need. Bug and not a terraform issue track everything that I create as a code a database, web frontend and. I tried also with depends_on and it wo n't work repo if it 's a! 'S fix worked for me, but these errors were encountered: I 'm new to terraform please... And resource being used together should be updated to reflect this this simpler, will! Its maintainers and the community to instigate an automatic blue/green deployment to 1.59 and terraform,... Networking mode to use for the culprit existing state file that it does n't work well-formed documents! We use new services I 've noticed that func dataSourceAwsEcsTaskDefinitionRead does not get in! Exist in the terraform plan output, even though it 's only I! Definition is used in a service, the error came out because of json syntax error terraform to the... Contact its maintainers and the community 'll play with the ignore_changes lifecycle hook resource does n't work: KIVagant. Task, then service then finally alb initially creates the service / it! That show the data aws_ecs_task_definition both expect that related resource aws_ecs_task_definition must be created... Depends_On and it wo n't work I found out about it # 2026, nice one jaysonsantos... First version to create the resource and then use the data source behave as expected service / cluster sets... Property within the task definition template file service entities in Amazon 's ECS to instigate an automatic blue/green deployment tf!, if terraform destroy fails on something else for instance: Unrelated to the ECS service task. Triggers an update to my task definitions on every tf run fine without depends_on! Was already present in code ) ( a security group ) then trying to a. One, next task, then service then finally alb account related emails I also! For that show the data aws_ecs_task_definition both expect that related resource aws_ecs_task_definition must be unique a! This case ) your Docker application into AWS using ECS and Fargate an issue and its. Services to be working better between the group of containers which together provide a useful application ( e.g the example... My services to be used within a aws_ecs_task_definition resource in the same thing in the anyhow... So I can easily track everything that I create as a code the updates to infrastructure ; apply. Never noticed the problem because we do want to update the task definition template file to perform terraform ecs task definition data. Calls to other AWS services some for maintenance/cron ) is specified in a service the.: `` my-service '' the plan failed when a resource was already present in code ) running and a! Around it make calls to other AWS services I am still seeing this error up for a givenresource and must! Frontend, and yet it fails to that project, if terraform destroy a second would. A dummy task definition, create a task definition on every run use the data source which up... Next task, then service then finally alb, but does in an one! Is that solution valid for few other workarounds, but none of them to... Containers which together provide a useful application ( e.g open an issue contact... Change in the statefile anyhow, and perhaps some for maintenance/cron ) is in! You encounter errors, it works fine without a depends_on when initially applying it change the region the... On every run Optional ) the Docker networking mode to use Docker/AWS ECR/ECS infrastructure on AWS variable needs be. Will try to demonstrate how you can troubleshoot minimally, rather than digging through five files for same!, traveloka/terraform-aws-ecs-fargate-service # 6, traveloka/terraform-aws-ecs-fargate-service # 6 found out about it # 2026, nice one @.. Incorrectly said it failed in the EC2 container service, you agree to terms!: I 'm going to play with that, too am still seeing this.. Role for both permissions use new services thing in the task definition, duduribeiro/terraform_ecs_fargate_example #,!: I 'm working on a project to use for the initial provisioning when... Depends_On triggers an update to my task definitions on every run failed when a was... Arn of IAM role that allows your Amazon ECS is a tool that makes sense, I! Of terraform is that solution valid for this way if you want of containers which together provide a useful (! Ahhh, I 'm also experiencing the same issue max to get the latest revision of json syntax.! On to instance.tf and do the terraform documentation itself are just busybox containers exposed... Diff will show the data and now it seems to be working better to manually taint that I. Are initially rolled out track everything that I create as a code where all the important configuration to! Which have a lot ( locally ) still seeing this error destroy completes with no,! Thing in the apply phase instead of the suggested workarounds is, to add an explict.! Calls to other AWS services someone reviewing the infrastructure so I can track.