data sources and outputs

[*]

Table of contents

  1. Day 1: Introduction

  2. Day 2: Raise the servers

  3. Day 3: Data sources and outputs

Links

Some links require VPN

aws CLI command reference

CLI command reference yc

Terraform provider aws

Terraform provider Yandex

Today we will get acquainted with the concepts data source and output; let’s see how the changes are applied to the already existing infrastructure.

data source

Last time, to raise the server, we had to search part of the parameters using the cli (or console), and specifically the subnet_id and id of the OS image. In order not to do this every time and in order to have up-to-date information (for example, the latest version of the OS), there is datasource. This entity makes it possible to receive information from the provider, which is in no way related to the current infrastructure (the one described in our file)

The syntax for all providers is the same, the same as for the resource:

data "<НАЗВАНИЕ>" "<ИМЯЮ>" {
	<ПАРАМЕТР> = <ЗНАЧЕНИЕ>
	<ДРУГОЙ_ПАРАМЕТР> = {
		<ПАРАМЕТР> = <ЗНАЧЕНИЕ>
	}
}

Imagine that the resource and the date are the same, but we ourselves initiate the resource, and the date is something that already exists.

Yandex

To get the id for Ubuntu 22.04, you must specify the family parameter – ubuntu-2204-lts; subnet name to get its id – default-ru-central1-a

yandex/main.tf

...
data "yandex_compute_image" "last_ubuntu" {
  family = "ubuntu-2204-lts"  # ОС (Ubuntu, 22.04 LTS)
}

data "yandex_vpc_subnet" "default_a" {
  name = "default-ru-central1-a"  # одна из дефолтных подсетей
}
...

aws

With aws, it’s a little more complicated: to get the id of the latest version of ubuntu, you need to specify the id of the owner of the image and its name (we got this through cli in the last part):

The asterisk replaces the version number, which we do not specify because we always want the latest version: this is the parameter most_recent = true

To get the subnet id, specify the desired zone (let it be like in Yandex – “a”)

The asterisk replaces the version number, which we do not specify because we always want the latest version: this is the parameter most_recent = true

To get the subnet id, specify the desired zone (let it be like in Yandex – “a”)

aws/main.tf

...
data "aws_ami" "last_ubuntu" {
  most_recent      = true # только новейшая версия
  owners           = ["099720109477"] # id владельца образа

  filter {
    name   = "name" # название фильтра - по какому параметру искать
    values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server*"] # что искать
  }
}

data "aws_subnet" "default_a" {
  filter {
    name   = "availability-zone"
    values = ["eu-north-1a"]
  }
}
...

Outputs and value substitution

Now you need to use this received data in place of hardcode. But, in addition to substituting them into resource parameters, we will display them in the terminal with output.

Conclusions (let’s call them that) are used not only to display information on the screen, but for now let’s stop there. Let’s add the id of the OS image and the id of the subnet to the output, as well as add the ip address of the raised server.

To get the desired values ​​from objects, you just need to refer to them by name through a dot.

Important: terraform reads all files with extension .tf in the directory as a whole – you can describe everything in one file or in several, as you like (for the same reason, the order in which entities are specified in the file itself is not important).

touch outputs.tf

Yandex

yandex/outputs.tf

output "default_instance_public_ip" {
    value = yandex_compute_instance.default.network_interface[0].nat_ip_address
}

output "subnet_id" {
    value = data.yandex_vpc_subnet.default_a.subnet_id
}

output "last_ubuntu" {
    value = data.yandex_compute_image.last_ubuntu.id
}

yandex/main.tf

terraform {
  required_providers {
    yandex = {
      source = "yandex-cloud/yandex"
    }
  }
}

provider "yandex" {
  token     = "..." # OAuth-токен яндекса
  cloud_id  = "b1gos1rh49bip4rnmrmg"
  folder_id = "b1gjju43i1pr11i5c4ic"
  zone      = "ru-central1-a"
}


data "yandex_compute_image" "last_ubuntu" {
  family = "ubuntu-2204-lts"  # ОС (Ubuntu, 22.04 LTS)
}

data "yandex_vpc_subnet" "default_a" {
  name = "default-ru-central1-a"  # одна из дефолтных подсетей
}


# ресурс "yandex_compute_instance" т.е. сервер
# Terraform будет знаеть его по имени "yandex_compute_instance.default"
resource "yandex_compute_instance" "default" { 
  name = "test-instance"
	platform_id = "standard-v1" # тип процессора (Intel Broadwell)

  resources {
    core_fraction = 5 # Гарантированная доля vCPU
    cores  = 2 # vCPU
    memory = 1 # RAM
  }

  boot_disk {
    initialize_params {
      image_id = data.yandex_compute_image.last_ubuntu.id
    }
  }

  network_interface {
    subnet_id = data.yandex_vpc_subnet.default_a.subnet_id
    nat = true # автоматически установить динамический ip
  }
}

aws

aws/outputs.tf

output "default_instance_public_ip" {
    value = aws_instance.default.public_ip
}

output "subnet_id" {
    value = data.aws_subnet.default_a.id
}

output "last_ubuntu" {
  value = data.aws_ami.last_ubuntu.id
}

aws/main.tf

provider "aws" {
  access_key = "AK..."
  secret_key = "2X..."
  region = "eu-north-1"
}

data "aws_ami" "last_ubuntu" {
  most_recent      = true
  owners           = ["099720109477"]

  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server*"]
  }
}

data "aws_subnet" "default_a" {
  filter {
    name   = "availability-zone"
    values = ["eu-north-1a"]
  }
}

#ресурс "aws_instance" т.е. сервер
#terraform будет знаеть его по имени "aws_instance.default"
resource "aws_instance" "default" {
  ami           = data.aws_ami.last_ubuntu.id # ОС (Ubuntu, 22.04 LTS)
  instance_type = "t3.micro" # тип процессора и ресурс машины (CPU и RAM)
	subnet_id = data.aws_subnet.default_a.id # одна из дефолтных подсетей
  associate_public_ip_address = true # автоматически установить динамический ip
	tags = {
    Name = "test-instance"
	}
}

Application and data output to the terminal

After terraform apply we will see something like this:

Outputs:

# yandex
default_instance_public_ip = "51.250.95.35"
last_ubuntu = "fd8v0s6adqu3ui3rsuap"
subnet_id = "e9bdgo95ucmut6r7pioq"

# aws
default_instance_public_ip = "16.16.64.183"
last_ubuntu = "ami-0a2b8744b4fe77f92"
subnet_id = "subnet-82b67deb"

If we look at the output of the command plan (same output apply before applying), we will see the following:

Changes to Outputs:

# yandex
  + default_instance_public_ip = (known after apply)
  + last_ubuntu                = "fd8v0s6adqu3ui3rsuap"
  + subnet_id                  = "e9bdgo95ucmut6r7pioq"

# aws
	+ default_instance_public_ip = (known after apply)
  + last_ubuntu                = "ami-0a2b8744b4fe77f92"
  + subnet_id                  = "subnet-82b67deb"

Returning to the beginning of this article: data source it’s something that already exists, so even before application, we know this data: last_ubuntu and subnet_id; Unlike default_instance_public_ip, which we receive from the raised server, which means that we will find out this data only after apply.

Infrastructure Changes

Before that, we created entities from scratch. Now let’s try to change the existing ones. For example, let’s add a parameter with an ssh key to our servers, and see what terraform offers.

In order not to insert the public key into the file (too long and we can change), we better use file() to substitute the entire contents of any file into a string.

Yandex

In Yandex, to add an ssh key, you need metadata. You also need to specify the user to which the key will give access: user ubuntu created by default.

yandex/main.tf

...
resource "yandex_compute_instance" "default" { 
  name = "test-instance"
  ...
  metadata = {
    ssh-keys = "ubuntu:${file("~/.ssh/id_rsa.pub")}"
  }
}

aws

In aws things are a bit more complicated – here the key is a separate entitywhich means we need to add one more resource: aws_key_pair and specify it as the parameter value for aws_instance.

aws/main.tf

...
resource "aws_instance" "default" {
  ...
  key_name = aws_key_pair.test_key.key_name
}

resource "aws_key_pair" "test_key" {
  key_name   = "test-key"
  public_key = "${file("~/.ssh/id_rsa.pub")}"
}

Applying Changes

terraform plan will show us the terrafrom action plan:

Yandex

Yandex’s behavior is a bit non-obvious. The fact is that by creating a virtual machine without an ssh key (what we had initially), and adding it later, terraform does not offer to recreate the machine. ~ update in-place means that the resource will be modified without being recreated.

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # yandex_compute_instance.default will be updated in-place
  ~ resource "yandex_compute_instance" "default" {
        id                        = "fhm3kp62eahete5kofke"
      ~ metadata                  = {
          + "ssh-keys" = <<-EOT
...

However, after adding the key, connection via it will be unavailable. It is necessary to recreate the machine so that ssh does not require a password, but only a key. For this we add parameter replace to command: apply -replace=yandex_compute_instance.default. We explicitly indicate what needs to be recreated. Terraform will show this – we will get a new address, which can be seen in Changes to outputs.

...
# yandex_compute_instance.default will be replaced, as requested
-/+ resource "yandex_compute_instance" "default" {
...
Changes to Outputs:
  ~ default_instance_public_ip = "51.250.95.35" -> (known after apply)
...

After confirming and recreating the server, we can connect to it via ssh.

yc compute instance list
+----------------------+---------------+---------------+---------+--------------+-------------+
|          ID          |     NAME      |    ZONE ID    | STATUS  | EXTERNAL IP  | INTERNAL IP |
+----------------------+---------------+---------------+---------+--------------+-------------+
| fhmpng88a49dihen141a | test-instance | ru-central1-a | RUNNING | 62.84.117.26 | 10.128.0.17 |
+----------------------+---------------+---------------+---------+--------------+-------------+

aws

In aws, everything works as expected – terraform itself offers to recreate the server. He also points out: what exactly makes him do this. In this case, this key_name – # forces replacement means that this parameter is the reason for re-creation.

Terraform will perform the following actions:

# aws_instance.default must be replaced
-/+ resource "aws_instance" "default" {
     ...
      + key_name                           = "test-key" # forces replacement
		 ...
# aws_key_pair.test_key will be created
+ resource "aws_key_pair" "test_key" {
...
Changes to Outputs:
  ~ default_instance_public_ip = "16.16.64.183" -> (known after apply)
...

Unfortunately we can’t just connect to a new aws server – we need to configure security group (we will do this in one of the following articles). But the server is running and a new key has been generated.

aws ec2 describe-instances \
	--region eu-north-1 \
	--query "Reservations[*].Instances[*].{Instance:InstanceId,KeyName:KeyName,Address:PublicIpAddress,Name:Tags[?Key=='Name']|[0].Value}" \
	--output table
----------------------------------------------------------------------------
|                             DescribeInstances                            |
+--------------+-----------------------+-----------------+-----------------+
|    Address   |       Instance        |     KeyName     |      Name       |
+--------------+-----------------------+-----------------+-----------------+
|  13.49.23.154|  i-075afc37d4775ccb0  |  test-key       |  test-instance  |
+--------------+-----------------------+-----------------+-----------------+

aws ec2 describe-key-pairs \
	--region eu-north-1 \
	--query="KeyPairs[*].{KeyName:KeyName,CreateTime:CreateTime}" \
	--output table
-------------------------------------------------
|               DescribeKeyPairs                |
+----------------------------+------------------+
|         CreateTime         |     KeyName      |
+----------------------------+------------------+
|  2022-08-28T11:07:42+00:00 |  test-key        |
+----------------------------+------------------+

Let’s destroy everything c terraform destroy

Conclusion

Data source is an indispensable tool for building a complex infrastructure. Outputs seem a bit useless at this point, but we’ll see another use for this feature later on.

Any questions please email v.valentinvolkov@gmail.com. I’ll be glad to help!

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *