A Serverless Application Model stack that persists the $XRP price to the XRPL every minute as a TrustLine. There are no servers, it is effectively a "smart contract" in Python for the XRPL.

Overview

xrpl-price-persist-oracle-sam

This is a XRPL Oracle that publishes external data into the XRPL.

This Oracle was inspired by XRPL-Labs/XRPL-Persist-Price-Oracle. Also see the DEV post.

This Oracle is coded in python and built as a Serverless Application Model (SAM).

An example testnet account.

Deploying to your AWS account

To deploy to your AWS Account use the aws-sam-cli. If you don't have aws-sam-cli installed, you can grab it from pip or follow the installation documentation.

pip install -U aws-sam-cli

After installing aws-sam-cli, you can Build and Deploy.

# zip our function along with it's requirements from requirements.txt
# this also makes a new template pointing at those zips
sam build 
# now it's built, you'll be prompted to deploy, do so interactively with:
sam deploy --guided

This will walk you through the deployment of the Oracle, name the stack input the parameters (the wallet seed parameter is NoEcho)

You may generate a Testnet wallet at the Testnet faucet: https://xrpl.org/xrp-testnet-faucet.html Click "Generate Testnet Credentials" and use the Secret as the input to the WalletSecret parameter.

Besides the one required parameter without a default (the wallet secret seed), you can accept all the defaults (including the stack name) and you'll be persisting aggregate prices to the XRPL Testnet. (An example testnet account.)

After deployment, you can tail the logs of the function like, where my-stack-name is what you chose to name your stack during the --guided deploy, if you chose the default it's sam-stack:

sam logs --stack-name my-stack-name -n OracleFunction -t

Note on NoEcho Cloudformation Parameter for Wallet Secret Seed

The NoEcho parameter of the wallet secret seed ensures that the parameter may not be read or may not be logged during any cloudformation events.

The produced resource that !Ref's the parameter in use, the function with it's environment variable. Be aware that any other user able to access the AWS account used to stand this stack will be able to read the secret on the lambda function itself.

If you're in a trusted account, and don't provide access to tools or services that would have access to these things you'll be fine.

Otherwise, you'll have a couple options:

One option is to encrypt the Lambda environment vars in transit (they're encrypted at rest by default). This would then require decrypting it in the function using a KMS call. (see Securing Environment Variables)

Alternatively you might want to call some key management service / secrets management service independently. There is two other AWS services that you could use, for managing encrypted secrets in transit.

One is Simple Systems Manager (ssm, specifically ssm-encrypted parameter store type). This includes some additional costs if the function needs to cold start (assuming you're persisting the client in the outer scope for subsequent executions). (see Sharing Secrets with AWS Lambda Using AWS Systems Manager Parameter Store)

Another is Secrets Manager aws-secrets-manager, which is also an additional cost. (see secretsmanager_basics.py)

There are many options! This is just a minimal example :)

Comments
  • New Oracle Address rEGGEgg… (

    New Oracle Address rEGGEgg… ("🥚🥚" / EGGE.gg)

    A new wallet address that is domain verified with the domain displayed in the vanity address.

    The domain is EGGE.gg - this feels more apt. :fried_egg:

    I wouldn't want someone to go to the domain, like "EGGD.gg" and be thrown into some scammy stuff because they saw a link in a vanity address. There is the context that .gg domains don't like that sort of stuff and they would be banned from registering, but that wouldn't be a great experience for the community.

    This will be the new address.

    rEGGEgg9hQSHLxUwycGnmjW2GKX9thn2yH
    

    This replaces the old address

    rEGGDggxupqxJ3ZbDTLUzKtpHxHyhtUtiU
    

    The rollout 🚜:

    • [x] 1. New stack created with new wallet and secret seed in a pull-request branch that overwrites mainnet stack workflow
      • This will be using the same code, but based on it's invocation time; it may produce different results as it will reflect the markets during it's invocation.
    • [x] 2. Announce change via twitter
    • [x] 3. Reach out to ShortTheFomo where the oracle is featured, to notify of change
    • [x] 4. Domain verify "🥚🥚" / with a EGGE.gg/.well-known/xrp-ledger.toml
    • [x] #39
      • NOTE this might not be possible if the previous trust lines count towards the ledger objects imagevia
        • if that's the case I'll black-hole the wallet, extracting what is possible 😙
    enhancement 
    opened by yyolk 2
  • try autofill

    try autofill

    https://xrpl-py.readthedocs.io/en/stable/source/xrpl.transaction.html?highlight=autofill#xrpl.transaction.safe_sign_and_autofill_transaction

    TIL about autofill :nerd_face:

    opened by yyolk 1
  • Move remaining funds from EGGD to EGGE - is AccountDelete possible?

    Move remaining funds from EGGD to EGGE - is AccountDelete possible?

    . After a brief period EGGE… inherits remaining EGGD… through a delete account transaction.

    • [x] move remaining funds

    • [x] is AccountDelete possible?

    • ~blackhole wallet if AccountDelete not possible~

    • NOTE this might not be possible if the previous trust lines count towards the ledger objects imagevia

      • if that's the case I'll black-hole the wallet, extracting what is possible 😙
    opened by yyolk 1
  • semicolon over comma

    semicolon over comma

    match csv delimiter of semicolon.

    i think this makes sense if an aggregate of currencies where some use commas like the EUR ( € 10 € 10,10 € 10 000,10)

    another wise decision from Wietse

    https://www.quora.com/How-do-you-write-sums-of-euros-over-1-000-i-e-where-do-you-put-the-full-stops-and-the-commas

    this also has the added benefit of matching the serialization of the XUMM oracle.

    opened by yyolk 1
  • use autofill instead of the base_fee

    use autofill instead of the base_fee

    remove fee set to base_fee of the ledger, defer to autofill's lookup:

    https://github.com/XRPLF/xrpl-py/blob/8b27082b71a987676970c7a403460c596d5feb6a/xrpl/asyncio/transaction/main.py#L183

    opened by yyolk 0
  • update rpc url for mainnet

    update rpc url for mainnet

    xrplcluster.com responds with a message when being throttled that creates an avalanche of failures.

    In the interim until my node is available, I may have to swap between json-rpc urls

    This should resolve the non-reporting gaps over the last 12 hours

    image image

    opened by yyolk 0
  • dont check_fee at autofill

    dont check_fee at autofill

    don't check_fee (default=True) which might be getting us throttled at our freq..

    on function invocation we cache this for subsequent re-use.

    from main, prior to this merge: https://github.com/yyolk/xrpl-price-persist-oracle/blob/4895a1f1484d53f85e527d4619415fbc4c48a044/oracle/contract.py#L45

    opened by yyolk 0
  • xrp-price-aggregate@0.0.15

    [email protected]

    this addresses a step missing in 0.0.14; it was non-breaking as the result was the same as the prior 0.0.13 release of any exceptions.

    the summary is identical to #43

    starts to address #27 - by filtering out at the time of failure any chains of providers requests. Platforms that respond to at least the initial request will be included in the aggregate results

    i.e.:

    [ ❌ ] <~ won't be included
    [ ✅ ✅ ❌ ] <~ will be included 👍 
    [ ✅ ❌] <~ will be included 👍
    [ ✅ ✅ ✅ ] <~ will be included 👍
    

    see yyolk/xrp-price-aggregate#20 https://github.com/yyolk/xrp-price-aggregate/releases/tag/0.0.15 for more info

    opened by yyolk 0
  • xrp-price-aggregate@0.0.14

    [email protected]

    [email protected]
    

    starts to address #27 - by filtering out at the time of failure any chains of providers requests. Platforms that respond to at least the initial request will be included in the aggregate results

    i.e.:

    [ ❌ ] <~ won't be included
    [ ✅ ✅ ❌ ] <~ will be included 👍 
    [ ✅ ❌] <~ will be included 👍
    [ ✅ ✅ ✅ ] <~ will be included 👍
    

    see yyolk/xrp-price-aggregate#19 and https://github.com/yyolk/xrp-price-aggregate/releases/tag/0.0.14 This is a broad stroke.

    opened by yyolk 0
  • update with old distribution using new paths

    update with old distribution using new paths

    see yyolk/xrpl-price-persist-oracle-metrics#1 - now using a more optimal on-demand graph fetch (still cached) that doesn't involve running a function every 5 minutes. :sunglasses:

    opened by yyolk 0
  • xrp-price-aggregate@0.0.13

    [email protected]

    closes #34

    https://github.com/yyolk/xrp-price-aggregate/releases/tag/0.0.13

    notably, threexrp.dev is temporarily dropped (see yyolk/xrp-price-aggregate#13)

    opened by yyolk 0
  • An abundance of JSONDecodeError

    An abundance of JSONDecodeError

    While using the JsonRPCClient; the oracle will see an abundance of JSONDecodeErrors that are coming as a result of the server saying it's too busy. image image

    I was aware of this happening, and was swallowing, notifying and then raising FailedExecutionWillRetry

    Got a JSONDecodeError of 'Expecting value: line 1 column 1 (char 0)'. Retrying the transaction by failing this execution.
    
    
    
    JSONDecodeError: Expecting value: line 1 column 1 (char 0)
    Traceback (most recent call last):
      File "/var/lang/lib/python3.9/importlib/__init__.py", line 127, in import_module
        return _bootstrap._gcd_import(name[level:], package, level)
      File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
      File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
      File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 850, in exec_module
      File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
      File "/var/task/contract.py", line 45, in <module>
        base_fee = get_fee(xrpl_client)
      File "/var/task/xrpl/ledger/main.py", line 54, in get_fee
        return asyncio.run(main.get_fee(client))
      File "/var/lang/lib/python3.9/asyncio/runners.py", line 44, in run
        return loop.run_until_complete(main)
      File "/var/lang/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
        return future.result()
      File "/var/task/xrpl/asyncio/ledger/main.py", line 64, in get_fee
        response = await client.request_impl(Fee())
      File "/var/task/xrpl/asyncio/clients/json_rpc_base.py", line 46, in request_impl
        return json_to_response(response.json())
      File "/var/task/httpx/_models.py", line 1415, in json
        return jsonlib.loads(self.text, **kwargs)
      File "/var/lang/lib/python3.9/json/__init__.py", line 346, in loads
        return _default_decoder.decode(s)
      File "/var/lang/lib/python3.9/json/decoder.py", line 337, in decode
        obj, end = self.raw_decode(s, idx=_w(s, 0).end())
      File "/var/lang/lib/python3.9/json/decoder.py", line 355, in raw_decode
        raise JSONDecodeError("Expecting value", s, err.value) from None
    
    opened by yyolk 0
  • Data sources

    Data sources

    Not a bug but a suggestion

    • i would look at changing up some of the data sources compared with xumm
    • there is value in having some over lap but your well over 50%

    My suggestion is as follows.

    • Bistamp and Kraken should def stay
    • FTX is mostly a derivatives exchange, but they do have the one fiat pair (keep that in mind from its data) it a good outsider.
    • Bitrue is USD via tether! It should have a lower weighting in the overall price, if you want to keep that in the price calculation.
    • add Bitso, cryptocom I am waiting on the new coinsph websocket to launch when that pro exchange fires up that def should go in there.
    • Lastly is phemex they provide most of the tech for all the fiat on ramps all exchanges use, this IMO is an important one.

    Rocking.

    opened by lathanbritz 3
  • 🏗️ fixing 🕳️ in data 🚧

    🏗️ fixing 🕳️ in data 🚧

    🕳️ in data as a result of a few causes that will be fixed soon :)

    • [ ] Most issues are caught by XRPLSendReliableTransaction - our SLA is 4 ledgers. This will be fine-tuned. 4 was indeed a good threshold if we were to retry manually, we may defer to xrpl-py (investigate)

    • [ ] terQueued sometimes we get queued! (but theres a probability we'll make it in on their behalf, we can't rely on it though) We see gaps in our data, but there might be some that made it through - this is the relevant lines: https://github.com/yyolk/xrpl-price-persist-oracle-sam/blob/9407737d760d172e8e7e732c4d8edc4e4b59ace0/oracle/contract.py#L231-L253

    • [ ] telInsufFee base fee of 10 💧, we sometimes miss for an overloaded ledger!

    • [ ] xrp-price-aggregate should be more robustful yyolk/xrp-price-aggregate#17

    • [ ] #29

    I do see this as a bug+enhancement, the end result most likely is we may we spend a bit more drops to ensure we're publishing every minute.

    bug enhancement 
    opened by yyolk 0
Owner
Joseph Chiocchi
🥚
Joseph Chiocchi
The Sue Gray Alert System was a 5 minute project that just beeps every time a new article is updated or published on Gov.UK's news pages.

The Sue Gray Alert System was a 5 minute project that just beeps every time a new article is updated or published on Gov.UK's news pages.

Dafydd 1 Jan 31, 2022
Powerful Ethereum Smart-Contract Toolkit

Heimdall Heimdall is an advanced and modular smart-contract toolkit which aims to make dealing with smart contracts on EVM based chains easier. Instal

Jonathan Becker 69 Dec 26, 2022
💻 A fully functional local AWS cloud stack. Develop and test your cloud & Serverless apps offline!

LocalStack - A fully functional local AWS cloud stack LocalStack provides an easy-to-use test/mocking framework for developing Cloud applications. Cur

LocalStack 45.3k Jan 2, 2023
An Amazon Price Tracker app helps you to buy which product you want within sale price by sending an E-Mail.

Amazon Price Tracker An Amazon Price Tracker app helps you to buy which product you want within sale price by sending an E-Mail. Installing Download t

Aytaç Kaşoğlu 2 Feb 10, 2022
This is a discord token generator(requests) which works and makes 200 tokens per minute

Discord Email verified token generator Creates email verified discord accounts (unlocked) Report Bug · Discord server Features Profile pictures and na

null 131 Dec 10, 2022
Senexia - A powerful telegram bot to manage your groups as effectively as possible

⚡ Kenechi bot ⚡ A Powerful, Smart And Simple Group Manager ... Written with AioG

Akhi 2 Jan 11, 2022
szrose is an all in one group management bot made for managing your group effectively with some advance security tools & Suit For All Your Needs ❤️

szrose is an all in one group management bot made for managing your group effectively with some advance security tools & Suit For All Your Needs ❤️

szsupunma 93 Jan 7, 2023
Price checker windows application

Price-Checker price checker windows application This application monitors the prices of selected products and displays a notification if the price has

Danila Tsareff 1 Nov 29, 2021
Use GitHub Actions to create a serverless service.

ActionServerless - Use GitHub Actions to create a serverless service ActionServerless is an action to do some computing and then generate a string/JSO

null 107 Oct 28, 2022
SQS + Lambda를 활용한 문자 메시지 및 이메일, Voice call 호출을 간단하게 구현하는 serverless 템플릿

AWS SQS With Lambda notification 서버 구축을 위한 Poc TODO serverless를 통해 sqs 관련 리소스(람다, sqs) 배포 가능한 템플릿 작성 및 배포 poc차원에서 간단한 rest api 호출을 통한 sqs fifo 큐에 메시지

김세환 4 Aug 8, 2021
GitHub action to deploy serverless functions to YandexCloud

YandexCloud serverless function deploy action Deploy new serverless function version (including function creation if it does not exist). Inputs yc_acc

Много Лосося 4 Apr 10, 2022
Tools used by Ada Health's internal IT team to deploy and manage a serverless Munki setup.

Serverless Munki This repository contains cross platform code to deploy a production ready Munki service, complete with AutoPkg, that runs entirely fr

Ada Health 17 Dec 5, 2022
Cookiecutter templates for Serverless applications using AWS SAM and the Rust programming language.

Cookiecutter SAM template for Lambda functions in Rust This is a Cookiecutter template to create a serverless application based on the Serverless Appl

AWS Samples 24 Nov 11, 2022
The AWS Lambda Serverless Blind XSS App

Ass The AWS Lambda Serverless Blind XSS App 利用VPS配置XSS平台太麻烦了,如果利用AWS的Lambda那不就是一个域名的事情么?剩下的环境配置、HTTPS证书、隐私性、VPS续费都不用管了, 所以根据xless重写了Lambda平台的XSS,利用sla

cocokey 2 Dec 27, 2021
ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete

ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete

null 240 Jan 6, 2023
“ Hey there 👋 I'm Daisy „ AI based Advanced Group Management Bot Suit For All Your Needs ❤️.. Source Code of @Daisyxbot

Project still under heavy development Everything will be changed in the release “ Hey there ?? I'm Daisy „ AI based Advanced telegram Group Management

TeamDaisyX 43 Nov 12, 2022
A project that alerts me when there's a dog outside so I can go look at it.

Dog Detector A project that alerts me when there's a dog outside so I can go look at it. Tech Specs This script uses the YOLOv3 object detection model

rydercalmdown 58 Jul 29, 2022