Quantcast
Channel: Recent Questions - Stack Overflow
Viewing all articles
Browse latest Browse all 12111

How I can fetch only specific PK from dynamodb to opensearch

$
0
0

So basically I'm trying to set up an open search sync pipeline on aws. Right now, I have the config for the pipeline as:

version: "2"dynamodb-pipeline:source:dynamodb:acknowledgments: truetables:  - table_arn: <>    stream:      start_position: LATEST    export:      s3_bucket: <>      s3_region: us-east-1      s3_prefix: <>aws:  sts_role_arn:<>  region: us-east-1sink:- opensearch:  hosts:    - https://dummy.amazonaws.com  index: index  template_content: |    {"settings":{"analysis":{"analyzer":{"autocomplete_analyzer":{"tokenizer":"standard","type":"custom","filter":["lowercase","autocomplete_filter"]}},"filter":{"autocomplete_filter":{"type":"edge_ngram","min_gram":2,"max_gram":20}}}},"mappings":{"properties":{"name":{"type":"text","analyzer":"autocomplete_analyzer","search_analyzer":"standard"},"summary":{"type":"text","analyzer":"autocomplete_analyzer","search_analyzer":"standard"},"category":{"type":"keyword"}}}}  index_type: custom  document_id: ${getMetadata("primary_key")}  action: ${getMetadata("opensearch_action")}  document_version: ${getMetadata("document_version")}  document_version_type: external  aws:    sts_role_arn: <role>    region: us-east-1

So I don't want everything from dynamodb, instead I want only PK with value GAMESEARCH to be indexed, I mean the entire dynamodb row record for matching PK, for which I need to specify the template for open search mapping.

Right now the above config kind of tries to copies everything from dynamodb, which I don't want.


Viewing all articles
Browse latest Browse all 12111

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>