Amazon Augmented AI Learning Lab¶
1. Overview¶
Amazon Augmented AI (Amazon A2I) is a service that brings human review of ML predictions to all developers by removing the heavy lifting associated with building human review systems or managing large numbers of human reviewers.
Many ML applications require humans to review low-confidence predictions to ensure the results are correct. For example, extracting information from scanned mortgage application forms can require human review due to low-quality scans or poor handwriting. Building human review systems can be time-consuming and expensive because it involves implementing complex processes or workflows, writing custom software to manage review tasks and results, and managing large groups of reviewers.
In this lab, we will learn how to build a robust, scalable, customizable Human In Loop system.
There are two major components:
Infrastructure
HIL Application
Terminology
A2I: Augmented AI, the AWS Service name
HIL: Human in Loop, a feature in A2I
Task Template:
2. Infrastructure - Create Labeling Workforce Private Team¶
Firstly, we want to create a labeling workforce private team and invite your employee to help us do the Human-in-Loop tasks. Then, the labeling workers can use the web app to log in their workspace and to do HIL tasks.
Amazon Sagemaker Ground Truth greatly simplifies the effort to create / manage this HIL GUI system. In tradition, without Amazon Sagemaker Ground Truth, you need to develop your own web application with GUI, and manage the login credential, data access policy, worker management your self.
go to AWS Sagemaker Console -> Ground Truth sub menu -> Labeling workforces -> Private -> Create Private Team:
configure the private team, you can follow detailed instruction below
Private Team Creation = Create a private team with AWS Cognito
Team details:
Team name =
my-labeling-team
Add Workers:
Invite new workers by email, Email address:
alice@example.com
Organization name:
my-org
Contact email:
admin@example.com
Enable Notifications: we don’t need this for learning.
Click “Create Private Team” button.
Now you can enter your Private Team console. There is a sign-un URL your workers can log in to the HIL workspace. If you want to add more workers to your team, You can invite more people by clicking the “Invite new workers” button.
As a worker, once you logged in to the HIL workspace, you will see the following GUI. Now we don’t have any HIL task available yet.
3. Infrastructure - Create Human in Loop Workflow¶
In this section, we will create all required AWS resources for Human in Loop Workflow, including:
An S3 bucket to store the HIL data
An IAM Role for Human Review Workflow execution
A Human Review Workflow definition that defines the metadata about this workflow
A Task template that defines the HIL task HTML UI
[20]:
# install additional python libraries
%pip install -r requirements.txt
Requirement already satisfied: pathlib_mate in /Users/sanhehu/venvs/python/3.8.11/dev_exp_share_venv/lib/python3.8/site-packages (from -r requirements.txt (line 1)) (1.0.3)
Requirement already satisfied: smart_open in /Users/sanhehu/venvs/python/3.8.11/dev_exp_share_venv/lib/python3.8/site-packages (from -r requirements.txt (line 2)) (5.2.1)
Requirement already satisfied: boto3 in /Users/sanhehu/venvs/python/3.8.11/dev_exp_share_venv/lib/python3.8/site-packages (from -r requirements.txt (line 3)) (1.22.9)
Requirement already satisfied: boto_session_manager in /Users/sanhehu/venvs/python/3.8.11/dev_exp_share_venv/lib/python3.8/site-packages (from -r requirements.txt (line 4)) (0.0.4)
Requirement already satisfied: s3pathlib in /Users/sanhehu/venvs/python/3.8.11/dev_exp_share_venv/lib/python3.8/site-packages (from -r requirements.txt (line 5)) (1.0.12)
ERROR: Could not find a version that satisfies the requirement box (from versions: none)
ERROR: No matching distribution found for box
WARNING: You are using pip version 21.2.4; however, version 22.2.2 is available.
You should consider upgrading via the '/Users/sanhehu/venvs/python/3.8.11/dev_exp_share_venv/bin/python -m pip install --upgrade pip' command.
Note: you may need to restart the kernel to use updated packages.
Below is an automation script. With a minimal parameter definition (the project name you want to use, provides AWS credential, etc …), you are able to deploy necessary AWS resources without writing code.
[3]:
# import standard library
import typing as T
import os
import json
import time
import uuid
import subprocess
import dataclasses
# import 3rd party library
from box import Box
from liquid import Template
from pathlib_mate import Path
from s3pathlib import S3Path, context
from boto_session_manager import BotoSesManager, AwsServiceEnum
dir_here = Path(os.getcwd()).absolute()
path_task_template = dir_here / "task.liquid"
path_task_ui_html = dir_here / "task.html"
@dataclasses.dataclass
class Lab:
# constant attributes
project_name: str = dataclasses.field()
labeling_team_arn: str = dataclasses.field()
bsm: BotoSesManager = dataclasses.field(default=None)
path_task_template: Path = dataclasses.field(default=path_task_template)
path_task_ui_html: Path = dataclasses.field(default=path_task_ui_html)
# derived attributes
s3_client: T.Any = dataclasses.field(default=None)
iam_client: T.Any = dataclasses.field(default=None)
sm_client: T.Any = dataclasses.field(default=None)
a2i_client: T.Any = dataclasses.field(default=None)
_workspace_portal_signin_url_cache: str = dataclasses.field(default=None)
def __post_init__(self):
if self.bsm is None:
self.bsm = BotoSesManager()
context.attach_boto_session(self.bsm.boto_ses)
self.s3_client = self.bsm.get_client(AwsServiceEnum.S3)
self.iam_client = self.bsm.get_client(AwsServiceEnum.IAM)
self.sm_client = self.bsm.get_client(AwsServiceEnum.SageMaker)
self.a2i_client = self.bsm.get_client(AwsServiceEnum.AugmentedAIRuntime)
@property
def project_name_slug(self) -> str:
return self.project_name.replace("_", "-")
@property
def common_tags(self) -> T.List[T.Dict[str, str]]:
return [
dict(
Key="ProjectName",
Value=self.project_name_slug,
)
]
# --- Create S3 bucket to store HIL data
@property
def bucket_name(self) -> str:
return f"{self.bsm.aws_account_id}-{self.bsm.aws_region}-{self.project_name_slug}"
@property
def bucket_console_url(self) -> str:
return f"https://s3.console.aws.amazon.com/s3/buckets/{self.bucket_name}?region={self.bsm.aws_region}&tab=objects"
def step_1a_create_s3_bucket(self) -> dict:
print("Create s3 bucket to store HIL data")
print(f" Preview at {self.bucket_console_url}")
# ref: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.create_bucket
response1 = self.s3_client.create_bucket(
Bucket=self.bucket_name,
CreateBucketConfiguration=dict(
LocationConstraint=self.bsm.aws_region,
),
)
# grant CORS permission so HIL UI can access artifacts in S3 bucket
# ref: https://docs.aws.amazon.com/sagemaker/latest/dg/sms-cors-update.html
# ref: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.put_bucket_cors
response2 = self.s3_client.put_bucket_cors(
Bucket=self.bucket_name,
CORSConfiguration={
"CORSRules": [
{
"AllowedHeaders": [],
"AllowedMethods": ["GET"],
"AllowedOrigins": ["*"],
"ExposeHeaders": ["Access-Control-Allow-Origin"],
}
]
},
)
print(f" Successful created s3://{self.bucket_name}")
return response1
def step_1b_delete_s3_bucket(self) -> dict:
print("Delete HIL data s3 bucket")
print(f" Preview at {self.bucket_console_url}")
s3dir = S3Path(self.bucket_name)
s3dir.delete_if_exists()
# ref: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.create_bucket
response = self.s3_client.delete_bucket(
Bucket=self.bucket_name,
)
print(f" Successful deleted s3://{self.bucket_name}")
return response
# --- Create IAM Role for HIL
@property
def flow_execution_role_name(self) -> str:
return f"{self.project_name_slug}-flow-role"
@property
def flow_execution_role_policy_name(self) -> str:
return f"{self.project_name_slug}-flow-role-in-line-policy"
@property
def flow_execution_role_arn(self) -> str:
return f"arn:aws:iam::{self.bsm.aws_account_id}:role/{self.flow_execution_role_name}"
@property
def flow_execution_role_console_url(self) -> str:
return f"https://console.aws.amazon.com/iamv2/home?region={self.bsm.aws_region}#/roles/details/{self.flow_execution_role_name}?section=permissions"
def step_2a_create_flow_execution_role(self) -> dict:
print("Create IAM role for Human review workflow")
print(f" Preview at {self.flow_execution_role_console_url}")
# ref: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/iam.html#IAM.Client.create_role
response1 = self.iam_client.create_role(
RoleName=self.flow_execution_role_name,
AssumeRolePolicyDocument=json.dumps({
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "sagemaker.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}),
Tags=self.common_tags,
)
# ref: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/iam.html#IAM.Client.put_role_policy
response2 = self.iam_client.put_role_policy(
RoleName=self.flow_execution_role_name,
PolicyName=self.flow_execution_role_policy_name,
PolicyDocument=json.dumps({
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject"
],
"Resource": [
f"arn:aws:s3:::{self.bucket_name}*"
]
}
]
}),
)
print(f" Successful created {self.flow_execution_role_arn}")
return response1
def step_2b_delete_flow_execution_role(self) -> dict:
print("Delete Human review workflow IAM role")
print(f" Preview at {self.flow_execution_role_console_url}")
# ref: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/iam.html#IAM.Client.delete_role_policy
response = self.iam_client.delete_role_policy(
RoleName=self.flow_execution_role_name,
PolicyName=self.flow_execution_role_policy_name,
)
# ref: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/iam.html#IAM.Client.delete_role
response = self.iam_client.delete_role(
RoleName=self.flow_execution_role_name,
)
print(f" Successful delete {self.flow_execution_role_arn}")
return response
# --- Create Task
@property
def task_template_name(self) -> str:
return f"{self.project_name_slug}"
@property
def task_template_arn(self) -> str:
return f"arn:aws:sagemaker:{self.bsm.aws_region}:{self.bsm.aws_account_id}:human-task-ui/{self.task_template_name}"
@property
def task_template_console_url(self) -> str:
return f"https://console.aws.amazon.com/a2i/home?region={self.bsm.aws_region}#/worker-task-templates/{self.task_template_name}"
def step_3a_create_hil_task_template(self) -> dict:
print("Create Human in Loop task template")
print(f" Preview at {self.task_template_console_url}")
# ref: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.create_human_task_ui
liquid_template = self.path_task_template.read_text(encoding="utf-8")
response = self.sm_client.create_human_task_ui(
HumanTaskUiName=self.task_template_name,
UiTemplate={
"Content": liquid_template
},
Tags=self.common_tags,
)
print(f" Successful created {self.task_template_name}")
return response
def step_3b_delete_hil_task_template(self) -> dict:
print("Delete Human in Loop task template")
print(f" Verify at {self.task_template_console_url}")
# ref: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.delete_human_task_ui
response = self.sm_client.delete_human_task_ui(
HumanTaskUiName=self.task_template_name
)
print(f" Successful delete {self.task_template_arn}")
return response
# --- Create Human review workflow
@property
def flow_definition_name(self) -> str:
return f"{self.project_name_slug}"
@property
def flow_definition_arn(self) -> str:
return f"arn:aws:sagemaker:{self.bsm.aws_region}:{self.bsm.aws_account_id}:flow-definition/{self.flow_definition_name}"
@property
def flow_definition_console_url(self) -> str:
return f"https://console.aws.amazon.com/a2i/home?region={self.bsm.aws_region}#/human-review-workflows/{self.flow_definition_name}"
@property
def s3dir_hil_input(self) -> S3Path:
return S3Path.from_s3_uri(f"s3://{self.bucket_name}/hil/input").to_dir()
@property
def s3dir_hil_output(self) -> S3Path:
return S3Path.from_s3_uri(f"s3://{self.bucket_name}/hil/output").to_dir()
def step_4a_create_flow_definition(self) -> dict:
print("Create Human review workflow definition, it may takes 30 sec ~ 1 minute")
print(f" Preview at {self.flow_definition_console_url}")
# ref: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.create_flow_definition
response = self.sm_client.create_flow_definition(
FlowDefinitionName=self.flow_definition_name,
HumanLoopConfig={
"WorkteamArn": self.labeling_team_arn,
"HumanTaskUiArn": self.task_template_arn,
"TaskTitle": self.task_template_name,
"TaskDescription": f"{self.task_template_name} description",
"TaskCount": 1, # if it
"TaskTimeLimitInSeconds": 3600,
},
OutputConfig={
"S3OutputPath": self.s3dir_hil_output.to_file().uri,
},
RoleArn=self.flow_execution_role_arn,
Tags=self.common_tags,
)
print(f" Successful created {self.flow_definition_arn}")
return response
def step_4b_delete_flow_definition(self) -> dict:
print("Delete Human review workflow definition, it may takes 30 sec ~ 1 minute")
print(f" Preview at {self.flow_definition_console_url}")
# ref: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#SageMaker.Client.delete_flow_definition
response = self.sm_client.delete_flow_definition(
FlowDefinitionName=self.flow_definition_name
)
print(f" Successful delete {self.flow_definition_arn}")
return response
# --- Start Human in Loop
@property
def labeling_workforce_console_url(self) -> str:
return (
f"https://{self.bsm.aws_region}.console.aws.amazon.com/sagemaker/"
f"groundtruth?region={self.bsm.aws_region}#/labeling-workforces"
)
@property
def workspace_portal_signin_url(self) -> str:
if self._workspace_portal_signin_url_cache is None:
self._workspace_portal_signin_url_cache = "https://" + self.sm_client.describe_workteam(
WorkteamName="my-labeling-team"
)["Workteam"]["SubDomain"]
return self._workspace_portal_signin_url_cache
def get_hil_console_url(self, hil_id: str) -> str:
return (
f"https://{self.bsm.aws_region}.console.aws.amazon.com/a2i/home?"
f"region={self.bsm.aws_region}#/human-review-workflows/"
f"{self.flow_definition_name}/human-loops/{hil_id}"
)
def start_human_loop(self, input_data: dict):
print("Start human loop ...")
print(f" You can enter the labeling portal from {self.workspace_portal_signin_url}")
response = self.a2i_client.start_human_loop(
HumanLoopName=str(uuid.uuid4()),
FlowDefinitionArn=self.flow_definition_arn,
HumanLoopInput={
"InputContent": json.dumps(input_data),
}
)
hil_arn = response["HumanLoopArn"]
hil_id = hil_arn.split("/")[-1]
hil_console_url = self.get_hil_console_url(hil_id)
print(f" You can preview HIL status at {hil_console_url}")
# --- Develop Task Template
def render_template_and_preview(self, input_data: dict):
# read liquid template
template = Template(self.path_task_template.read_text())
# convert task data to box, so it support dot notation
task = Box({"input": input_data})
# render template
content = template.render(task=task)
# write template to html file
self.path_task_ui_html.write_text(content)
# open html in browser
subprocess.run(["open", self.path_task_ui_html.abspath])
Now let’s run this automation script.
Firstly, you have to:
give a
project_name
, this will become a common prefix in all AWS resources naming convention.copy and paste the labeling team ARN that you created in the previous step to here, you can find the ARN in labeling team console
give this script necessary AWS credential using boto_session_manager library.
Then uncomment the following code line by line and execute:
lab.step_1a_create_s3_bucket()
lab.step_2a_create_flow_execution_role()
lab.step_3a_create_hil_task_template()
lab.step_4a_create_flow_definition()
[4]:
lab = Lab(
project_name="a2i-poc",
labeling_team_arn="arn:aws:sagemaker:us-east-2:669508176277:workteam/private-crowd/my-labeling-team",
# bsm=BotoSesManager(), # use this if you use AWS Sagemaker Notebook Instance
bsm=BotoSesManager(profile_name="aws_data_lab_sanhe_us_east_2"), # use this is you are on your local laptop
)
[21]:
# lab.step_1a_create_s3_bucket()
# lab.step_1b_delete_s3_bucket()
Create s3 bucket to store HIL data
Preview at https://s3.console.aws.amazon.com/s3/buckets/669508176277-us-east-2-a2i-poc?region=us-east-2&tab=objects
---------------------------------------------------------------------------
BucketAlreadyOwnedByYou Traceback (most recent call last)
Input In [21], in <cell line: 1>()
----> 1 lab.step_1a_create_s3_bucket()
Input In [3], in Lab.step_1a_create_s3_bucket(self)
74 print(f" Preview at {self.bucket_console_url}")
75 # ref: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.create_bucket
---> 76 response1 = self.s3_client.create_bucket(
77 Bucket=self.bucket_name,
78 CreateBucketConfiguration=dict(
79 LocationConstraint=self.bsm.aws_region,
80 ),
81 )
83 # grant CORS permission so HIL UI can access artifacts in S3 bucket
84 # ref: https://docs.aws.amazon.com/sagemaker/latest/dg/sms-cors-update.html
85 # ref: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.put_bucket_cors
86 response2 = self.s3_client.put_bucket_cors(
87 Bucket=self.bucket_name,
88 CORSConfiguration={
(...)
97 },
98 )
File ~/venvs/python/3.8.11/dev_exp_share_venv/lib/python3.8/site-packages/botocore/client.py:415, in ClientCreator._create_api_method.<locals>._api_call(self, *args, **kwargs)
412 raise TypeError(
413 "%s() only accepts keyword arguments." % py_operation_name)
414 # The "self" in this scope is referring to the BaseClient.
--> 415 return self._make_api_call(operation_name, kwargs)
File ~/venvs/python/3.8.11/dev_exp_share_venv/lib/python3.8/site-packages/botocore/client.py:745, in BaseClient._make_api_call(self, operation_name, api_params)
743 error_code = parsed_response.get("Error", {}).get("Code")
744 error_class = self.exceptions.from_code(error_code)
--> 745 raise error_class(parsed_response, operation_name)
746 else:
747 return parsed_response
BucketAlreadyOwnedByYou: An error occurred (BucketAlreadyOwnedByYou) when calling the CreateBucket operation: Your previous request to create the named bucket succeeded and you already own it.
[ ]:
lab.step_2a_create_flow_execution_role()
# lab.step_2b_delete_flow_execution_role()
[ ]:
lab.step_3a_create_hil_task_template()
# lab.step_3b_delete_hil_task_template()
[ ]:
lab.step_4a_create_flow_definition()
# lab.step_4b_delete_flow_definition()
Now HIL infrastructure are all deployed, we can start solving some sample business problems with HIL.
4. Learn A2I from Examples¶
The AWS A2I team has a public AWS Repo amazon-a2i-sample-task-uis. It has many HIL task HTML template, but without sample data (text / pdf / image / audio / video files). You still need to write many codes and set up your AWS console to run those samples.
In this tutorial, we provide the automation script, so you can focus on the.
How to Use
In current directory, there is a usecases
folder with many sub folders. Each sub folder represents a single use case. Belows are many Jupyter Notebook cells that each one represents the experimental script for a single use case. Each cell includes the following logics:
Update the task template to the chosen use case
Upload necessary artifacts to S3 bucket
Start the HIL
Show helpful information so you can inspect, preview your HIL task
Image Use Case¶
Bounding Box¶
[57]:
def bounding_box_use_case():
lab.path_task_template = dir_here / "usecases" / "images" / "bounding-box" / "task.liquid"
path_artifact = dir_here / "usecases" / "images" / "bounding-box" / "cat-and-dog.jpeg"
s3path_artifact = lab.s3dir_hil_input / "cat-and-dog.jpeg"
s3path_artifact.upload_file(path_artifact.abspath, overwrite=True)
print(f"Preview artifacts at {s3path_artifact.console_url}")
input_data = {
"taskObject": s3path_artifact.uri
}
lab.step_3b_delete_hil_task_template()
lab.step_3a_create_hil_task_template()
time.sleep(3)
lab.start_human_loop(input_data)
bounding_box_use_case()
Preview artifacts at https://console.aws.amazon.com/s3/object/669508176277-us-east-2-a2i-poc?prefix=hil/input/cat-and-dog.jpeg
Delete Human in Loop task template
Verify at https://console.aws.amazon.com/a2i/home?region=us-east-2#/worker-task-templates/a2i-poc
Successful delete arn:aws:sagemaker:us-east-2:669508176277:human-task-ui/a2i-poc
Create Human in Loop task template
Preview at https://console.aws.amazon.com/a2i/home?region=us-east-2#/worker-task-templates/a2i-poc
Successful created a2i-poc
Start human loop ...
You can enter the labeling portal from https://3zqu42gydr.labeling.us-east-2.sagemaker.aws
You can preview HIL status at https://us-east-2.console.aws.amazon.com/a2i/home?region=us-east-2#/human-review-workflows/a2i-poc/human-loops/ac04a258-7452-46c5-8f01-39bbfa94c842
Bounding Box Hierarchical¶
[13]:
def bounding_box_hierarchical_use_case():
lab.path_task_template = dir_here / "usecases" / "images" / "bounding-box-hierarchical" / "task.liquid"
path_artifact = dir_here / "usecases" / "images" / "bounding-box-hierarchical" / "lisa.png"
s3path_artifact = lab.s3dir_hil_input / "cat-and-dog.jpeg"
s3path_artifact.upload_file(path_artifact.abspath, overwrite=True)
print(f"Preview artifacts at {s3path_artifact.console_url}")
input_data = {
"taskObject": s3path_artifact.uri
}
lab.step_3b_delete_hil_task_template()
lab.step_3a_create_hil_task_template()
time.sleep(3)
lab.start_human_loop(input_data)
bounding_box_hierarchical_use_case()
Preview artifacts at https://console.aws.amazon.com/s3/object/669508176277-us-east-2-a2i-poc?prefix=hil/input/cat-and-dog.jpeg
Delete Human in Loop task template
Verify at https://console.aws.amazon.com/a2i/home?region=us-east-2#/worker-task-templates/a2i-poc
Successful delete arn:aws:sagemaker:us-east-2:669508176277:human-task-ui/a2i-poc
Create Human in Loop task template
Preview at https://console.aws.amazon.com/a2i/home?region=us-east-2#/worker-task-templates/a2i-poc
Successful created a2i-poc
Start human loop ...
You can enter the labeling portal from https://3zqu42gydr.labeling.us-east-2.sagemaker.aws
You can preview HIL status at https://us-east-2.console.aws.amazon.com/a2i/home?region=us-east-2#/human-review-workflows/a2i-poc/human-loops/a1e4dcbc-074f-4c8e-985b-1914055bc71d
Text Use Case¶
Key Phrase Extraction¶
[34]:
def key_phrase_extraction_use_case():
lab.path_task_template = dir_here / "usecases" / "text" / "key-phrase-extraction" / "task.liquid"
input_data = {
"taskObject": "Excellent bag and fast shipping! Bag arrived right on time and packaged very well. The bag itself is good quality! Was a bit skeptical ordering this bag off amazon but it's 100% authentic and the best price!"
}
lab.step_3b_delete_hil_task_template()
lab.step_3a_create_hil_task_template()
time.sleep(3)
lab.start_human_loop(input_data)
key_phrase_extraction_use_case()
Delete Human in Loop task template
Verify at https://console.aws.amazon.com/a2i/home?region=us-east-2#/worker-task-templates/a2i-poc
Successful delete arn:aws:sagemaker:us-east-2:669508176277:human-task-ui/a2i-poc
Create Human in Loop task template
Preview at https://console.aws.amazon.com/a2i/home?region=us-east-2#/worker-task-templates/a2i-poc
Successful created a2i-poc
Start human loop ...
You can enter the labeling portal from https://us-east-2.console.aws.amazon.com/sagemaker/groundtruth?region=us-east-2#/labeling-workforces
You can preview HIL status at https://us-east-2.console.aws.amazon.com/a2i/home?region=us-east-2#/human-review-workflows/a2i-poc/human-loops/30cf4969-8ec3-4db0-98b1-fe9edd7d9d1a
ML Use Case¶
Credit Card Application¶
You are a financial bank. You have a ML model that can automatically approve / deny credit card application. However, you want a HIL to review application cases that has high credit line or low ML confidence score.
Sample application and ML predict looks like this:
[18]:
def credit_card_application_use_case():
lab.path_task_template = dir_here / "usecases" / "ml" / "credit-card-application" / "task.liquid"
input_data = {
"application": [
{"key": "application_id", "value": "a-127"},
{"key": "application_date", "value": "2022-01-03"},
{"key": "ssn", "value": "111-22-3333"},
{"key": "name", "value": "James Bond"},
{"key": "month_income", "value": 15000},
],
"ml_prediction": "approve",
}
lab.step_3b_delete_hil_task_template()
lab.step_3a_create_hil_task_template()
time.sleep(3)
lab.start_human_loop(input_data)
credit_card_application_use_case()
Delete Human in Loop task template
Verify at https://console.aws.amazon.com/a2i/home?region=us-east-2#/worker-task-templates/a2i-poc
Successful delete arn:aws:sagemaker:us-east-2:669508176277:human-task-ui/a2i-poc
Create Human in Loop task template
Preview at https://console.aws.amazon.com/a2i/home?region=us-east-2#/worker-task-templates/a2i-poc
Successful created a2i-poc
Start human loop ...
You can enter the labeling portal from https://3zqu42gydr.labeling.us-east-2.sagemaker.aws
You can preview HIL status at https://us-east-2.console.aws.amazon.com/a2i/home?region=us-east-2#/human-review-workflows/a2i-poc/human-loops/cb60d9d9-462f-4bcb-be23-9944f12a1083
5. Develop Your Own Task Template¶
In the previous section, we deployed the task.liquid
file as a HIL Task Template to AWS, then we triggered a sample HIL task to preview how it looks it in web browser. In other word, we have to create lots AWS Resources for a simple test, including Labeling Workforce, S3 bucket, IAM Role, Flow Definition, Task Template, etc … This is a common pain point for many Amazon Augmented AI users.
In this Lab, we create a Python tool that can render the Task Template HTML with input data locally without triggering a HIL. Then you just need to focus on template development and data manipulation.
A2I uses the HTML template language liquid, an HTML template language created by Shopify. We use python-liquid library to render the HTML locally.
You can find useful resources to learn how to write a good Task Template below:
Tutorial: Create Custom Worker Task Templates: learn the basic concept about A2I task template.
Crowd HTML Elements Reference: all available crowd HTML tag usage.
amazon-a2i-sample-task-uis: get inspired by samples
Please review this code snippet to learn how to develop Task Template locally.
[19]:
dir_use_case = dir_here / "usecases" / "ml" / "credit-card-application"
# tell the tool where is the task template liquid file
lab.path_task_template = dir_use_case / "task.liquid"
# tell the tool where you want to store the rendered HTML
lab.path_task_ui_html = dir_use_case / "task.html"
# define the input data that matches your task template
input_data = {
"application": [
{"key": "application_id", "value": "a-127"},
{"key": "application_date", "value": "2022-01-03"},
{"key": "ssn", "value": "111-22-3333"},
{"key": "name", "value": "James Bond"},
{"key": "month_income", "value": 15000},
],
"ml_prediction": "approve",
}
# run this magic python function to render and preview
lab.render_template_and_preview(input_data)
[ ]: