Apache Libcloud is a Python library which hides differences between different cloud provider APIs and allows you to manage different cloud resources through a unified and easy to use API

Overview

Apache Libcloud - a unified interface for the cloud

Apache Libcloud is a Python library which hides differences between different cloud provider APIs and allows you to manage different cloud resources through a unified and easy to use API.

https://img.shields.io/badge/docs-latest-brightgreen.svg?style=flat https://github.com/apache/libcloud/workflows/CI/badge.svg?branch=trunk https://github.com/apache/libcloud/workflows/Publish%20pricing.json%20to%20S3%20bucket/badge.svg?branch=trunk https://bestpractices.coreinfrastructure.org/projects/152/badge
Code: https://github.com/apache/libcloud
License: Apache 2.0; see LICENSE file
Issues: https://issues.apache.org/jira/projects/LIBCLOUD/issues
Website: https://libcloud.apache.org/
Documentation: https://libcloud.readthedocs.io
Supported Python Versions: Python >= 3.5, PyPy 3 (Python 2.7 and Python 3.4 is supported by the v2.8.x release series)

Resources you can manage with Libcloud are divided into the following categories:

  • Compute - Cloud Servers and Block Storage - services such as Amazon EC2 and Rackspace Cloud Servers (libcloud.compute.*)
  • Storage - Cloud Object Storage and CDN - services such as Amazon S3 and Rackspace CloudFiles (libcloud.storage.*)
  • Load Balancers - Load Balancers as a Service, LBaaS (libcloud.loadbalancer.*)
  • DNS - DNS as a Service, DNSaaS (libcloud.dns.*)
  • Container - Container virtualization services (libcloud.container.*)

Apache Libcloud is an Apache project, see <http://libcloud.apache.org> for more information.

Documentation

Documentation can be found at <https://libcloud.readthedocs.org>.

Note on Python Version Compatibility

Libcloud v3.0.0 dropped support for Python 2.7 and Python 3.4 and now only supports Python >= 3.5.

If you still need to us Libcloud with one of the now unsupported versions, you can do that by using the latest release of Libcloud which still supports those versions (Libcloud v2.8).

Feedback

Please send feedback to the mailing list at <[email protected]>, or Github repo at <https://github.com/apache/libcloud/issues>.

Contributing

For information on how to contribute, please see the Contributing chapter in our documentation <https://libcloud.readthedocs.org/en/latest/development.html#contributing>

License

Apache Libcloud is licensed under the Apache 2.0 license. For more information, please see LICENSE and NOTICE file.

Comments
  • Scaleway Rebased

    Scaleway Rebased

    Description

    Revisiting #777 by rebasing it against the latest trunk. From its description:

    This is a simple/pure compute driver for https://www.scaleway.com/

    See also https://issues.apache.org/jira/browse/LIBCLOUD-956

    Status

    done, ready for review

    Checklist (tick everything that applies)

    • [x] Code linting (required, can be done after the PR checks)
    • [x] Documentation
    • [x] Tests
    • [x] ICLA (required for bigger changes)
    opened by danhunsaker 56
  • [Libcloud-556] Azure Compute Driver Support

    [Libcloud-556] Azure Compute Driver Support

    We have completed support for the Azure compute driver. This adds support for the core compute driver functions against Azure. The JIRA ticket can be found here:

    https://issues.apache.org/jira/browse/LIBCLOUD-556?jql=project%20%3D%20LIBCLOUD

    Let me know if there are any issues or if it can be accepted or not.

    Thanks. -matt

    opened by baldwinSPC 26
  • [LIBCLOUD-943] Add the option to Azure ARM ex_create_network_interfac…

    [LIBCLOUD-943] Add the option to Azure ARM ex_create_network_interfac…

    [LIBCLOUD-943] Add the option to Azure ARM ex_create_network_interface to create a static IP address

    Description

    https://issues.apache.org/jira/browse/LIBCLOUD-943

    Azure ARM driver's ex_create_network_interface is missing the option to create a static IP address. "privateIPAllocationMethod" is set to "Dynamic" in the "data" dictionary. This change adds the option to create the network interface with a static IP address.

    The value for the static IP address is defined with private_ip_address="n.n.n.n" in the ex_create_network_interface call.

    Status

    • done, ready for review

    Checklist (tick everything that applies)

    • [x] Code linting (required, can be done after the PR checks)
    • [x] Documentation
    • [ ] Tests
    • [ ] ICLA (required for bigger changes)
    status: needs tests 
    opened by aki-k 25
  • LIBCLOUD-781 Introducing Container-as-a-Service cloud as an API base driver to libcloud

    LIBCLOUD-781 Introducing Container-as-a-Service cloud as an API base driver to libcloud

    The intention of this PR is to enable support for pure container hosts such as docker, rkt as well as support the introduction of container-as-a-service providers.

    opened by tonybaloney 25
  • [LIBCLOUD-827] Add support for automatic SNI

    [LIBCLOUD-827] Add support for automatic SNI

    827 - Add Support for automatic SNI

    Description

    Adds support for automatic SNI using the hostname supplied to connect to.

    Python SSL has had SNI support for around six years (https://hg.python.org/cpython/rev/846c0e1342d0/)

    We need to convert old ssl socket wrap into a context to effect this.

    Status

    • done, ready for review

    Checklist (tick everything that applies)

    • [X] Code linting (required, can be done after the PR checks)
    • [N/A] Documentation
    • [X] Tests
    • [N/A] ICLA (required for bigger changes)
    opened by lochiiconnectivity 21
  • [LIBCLOUD-728] Add SSLError to retry decorator exceptions

    [LIBCLOUD-728] Add SSLError to retry decorator exceptions

    Add ssl.SSLError to the list of exceptions that libcloud.utils.misc.retry catches, since that SSLError can be a transient error. Also:

    • Clean up retry a bit, fixing an issue in which keyword arguments defaults are not applied
    • Enable retries in tests that are designed to test retries (previous, some tests passed trivially)

    See: https://issues.apache.org/jira/browse/LIBCLOUD-728

    opened by thesquelched 21
  • [LIBCLOUD-1017] Improvements to Azure ARM compute driver

    [LIBCLOUD-1017] Improvements to Azure ARM compute driver

    [LIBCLOUD-1017] Improvements to Azure ARM compute driver

    Description

    The following are the proposed changes in brief:

    1. Add functionality to resize a volume in Azure
    2. Add functionality to update the network profile of a node
    3. Add functionality to update a network interface's properties
    4. Add functionality to check IP address availability
    5. Create and Update some fixtures
    6. Some minor fixes to docstrings

    Status

    • done, ready for review

    Checklist (tick everything that applies)

    • [x] Code linting (required, can be done after the PR checks)
    • [x] Documentation
    • [x] Tests
    • [x] ICLA (required for bigger changes)
    api: compute drivers: azure 
    opened by palashgandhi 19
  • Add policy management methods to the ELB driver

    Add policy management methods to the ELB driver

    @Kami I am not able to debug the elb.py individually. I have followed the testing guidlines from the official documentation.

    P.S. From now onward i will create pull request after i have made few changes. I am not used to it, because in fedora community we used to discuss things on fpaste and then create pr.

    opened by rakontuh 19
  • Authenticate with Azure Ad for Provider AZURE_BLOBS

    Authenticate with Azure Ad for Provider AZURE_BLOBS

    Authenticate with Azure Ad for Provider AZURE_BLOBS

    Description

    Add the ability to authenticate Azure Blob Storage Accounts via Azure AD service principal secrets and certificates. This feature would be following this feature from jclouds (https://issues.apache.org/jira/browse/JCLOUDS-1558) It would be using the rest api to access azure ad

    For more information on contributing, please see Contributing section of our documentation.

    Status

    • done, ready for review

    Checklist (tick everything that applies)

    • [x] Code linting (required, can be done after the PR checks)
    • [x] Documentation
    • [x] Tests
    • [ ] ICLA (required for bigger changes)
    drivers: azure drivers: azure blobs 
    opened by brookewhite9 18
  • Libcloud 868 pdns 4

    Libcloud 868 pdns 4

    adapted methods to PowerDNS 4.0.1 REST API

    Description

    Changed create_record, update_record, to_records, _to_record to reflect changes in REST API specifications

    Status

    • done, ready for review

    Checklist

    Tested vs latest PDNS version 4.0.1 on a ubuntu server 16.10. It was installed from ubuntu repository

    stale 
    opened by biggosh 18
  • GKE Driver Functional

    GKE Driver Functional

    WIP in GKE

    Description

    Creating the google container driver

    Status

    • work in progress

    Checklist (tick everything that applies)

    • [x] Code linting (required, can be done after the PR checks)
    • [X] Documentation
    • [x] Tests
    • [x] ICLA (required for bigger changes)
    opened by andymaheshw 17
  • Bump coverage from 6.5.0 to 7.0.1

    Bump coverage from 6.5.0 to 7.0.1

    Bumps coverage from 6.5.0 to 7.0.1.

    Changelog

    Sourced from coverage's changelog.

    Version 7.0.1 — 2022-12-23

    • When checking if a file mapping resolved to a file that exists, we weren't considering files in .whl files. This is now fixed, closing issue 1511_.

    • File pattern rules were too strict, forbidding plus signs and curly braces in directory and file names. This is now fixed, closing issue 1513_.

    • Unusual Unicode or control characters in source files could prevent reporting. This is now fixed, closing issue 1512_.

    • The PyPy wheel now installs on PyPy 3.7, 3.8, and 3.9, closing issue 1510_.

    .. _issue 1510: nedbat/coveragepy#1510 .. _issue 1511: nedbat/coveragepy#1511 .. _issue 1512: nedbat/coveragepy#1512 .. _issue 1513: nedbat/coveragepy#1513

    .. _changes_7-0-0:

    Version 7.0.0 — 2022-12-18

    Nothing new beyond 7.0.0b1.

    .. _changes_7-0-0b1:

    Version 7.0.0b1 — 2022-12-03

    A number of changes have been made to file path handling, including pattern matching and path remapping with the [paths] setting (see :ref:config_paths). These changes might affect you, and require you to update your settings.

    (This release includes the changes from 6.6.0b1 <changes_6-6-0b1_>_, since 6.6.0 was never released.)

    • Changes to file pattern matching, which might require updating your configuration:

      • Previously, * would incorrectly match directory separators, making precise matching difficult. This is now fixed, closing issue 1407_.

      • Now ** matches any number of nested directories, including none.

    • Improvements to combining data files when using the

    ... (truncated)

    Commits
    • c5cda3a docs: releases take a little bit longer now
    • 9d4226e docs: latest sample HTML report
    • 8c77758 docs: prep for 7.0.1
    • da1b282 fix: also look into .whl files for source
    • d327a70 fix: more information when mapping rules aren't working right.
    • 35e249f fix: certain strange characters caused reporting to fail. #1512
    • 152cdc7 fix: don't forbid plus signs in file names. #1513
    • 31513b4 chore: make upgrade
    • 873b059 test: don't run tests on Windows PyPy-3.9
    • 5c5caa2 build: PyPy wheel now installs on 3.7, 3.8, and 3.9. #1510
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    external dependency python 
    opened by dependabot[bot] 0
  • Bump isort[pyproject] from 5.10.1 to 5.11.4

    Bump isort[pyproject] from 5.10.1 to 5.11.4

    Bumps isort[pyproject] from 5.10.1 to 5.11.4.

    Release notes

    Sourced from isort[pyproject]'s releases.

    5.11.4

    Changes

    :package: Dependencies

    5.11.3

    Changes

    :beetle: Fixes

    :construction_worker: Continuous Integration

    v5.11.3

    Changes

    :beetle: Fixes

    :construction_worker: Continuous Integration

    5.11.2

    Changes

    5.11.1

    Changes December 12 2022

    ... (truncated)

    Changelog

    Sourced from isort[pyproject]'s changelog.

    5.11.4 December 21 2022

    5.11.3 December 16 2022

    5.11.2 December 12 2022

    5.11.1 December 12 2022

    5.11.0 December 12 2022

    Commits
    • 98390f5 Merge pull request #2059 from PyCQA/version/5.11.4
    • df69a05 Bump version 5.11.4
    • f9add58 Merge pull request #2058 from PyCQA/deps/poetry-1.3.1
    • 36caa91 Bump Poetry 1.3.1
    • 3c2e2d0 Merge pull request #1978 from mgorny/toml-test
    • 45d6abd Remove obsolete toml import from the test suite
    • 3020e0b Merge pull request #2057 from mgorny/poetry-install
    • a6fdbfd Stop installing documentation files to top-level site-packages
    • ff306f8 Fix tag template to match old standard
    • 227c4ae Merge pull request #2052 from hugovk/main
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    external dependency python 
    opened by dependabot[bot] 0
  • Bump pylint from 2.15.8 to 2.15.9

    Bump pylint from 2.15.8 to 2.15.9

    Bumps pylint from 2.15.8 to 2.15.9.

    Commits
    • 1ded4d0 Bump pylint to 2.15.9, update changelog (#7952)
    • 785c629 [testutil] More information in output for functional test fail (#7948)
    • 3c3ab98 [pypy3.8] Disable multiple-statements false positive on affected functional t...
    • dca3940 Fix inconsistent argument exit code when argparse exit with its own error cod...
    • 494e514 Fix ModuleNotFoundError when using pylint_django (#7940) (#7941)
    • 83668de fix: bump dill to >= 0.3.6, prevents tests hanging with python3.11 (#7918)
    • eadc308 [github actions] Fix enchant's install in the spelling job
    • 391323e Avoid hanging forever after a parallel job was killed (#7834) (#7930)
    • 4655b92 Prevent used-before-assignment in pattern matching with a guard (#7922) (#7...
    • See full diff in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    external dependency python 
    opened by dependabot[bot] 0
  • Azure arm driver does not list all nodes

    Azure arm driver does not list all nodes

    Summary

    The Azure ARM driver only lists 50 nodes in a subscription.

    Detailed Information

    The function list_nodes() of the Azure ARM driver only lists 50 nodes when the Azure Portal shows more than 50 VMs.

    This is probably caused by missing pagination here: https://github.com/apache/libcloud/blob/trunk/libcloud/compute/drivers/azure_arm.py#L474

    According to the documentation, there should be a property nextLink in the response: https://learn.microsoft.com/en-us/rest/api/compute/virtual-machines/list-all

    Libcloud version: 3.6.1 Python version: 3.10 Operating system: Ubuntu 20.04

    opened by jan-mue 0
  • Add support for af south 1 in s3 driver

    Add support for af south 1 in s3 driver

    Add support for AWS af-south-1 region to S3 storage driver

    Description

    We noticed the following error when running our Ansible automation on af-south-1 VMs on AWS:

    File "/usr/local/lib/python3.8/dist-packages/libcloud/storage/drivers/s3.py", line 1134, in __init__
            raise ValueError('Invalid or unsupported region: %s' % (region))
        ValueError: Invalid or unsupported region: af-south-1
    

    The reason for the error is missing configuration for the af-south-1 region of AWS in the S3 driver of libcloud, which we have remediated locally and now raising this PR to contribute back to the project.

    Status

    • done, ready for review (Used pylint for linting)

    Checklist (tick everything that applies)

    • [x] Code linting (required, can be done after the PR checks)
    • [ ] Documentation
    • [ ] Tests
    • [ ] ICLA (required for bigger changes)
    opened by alier350 0
  • Azure Blob Storage get_object_cdn_url can't handle files with spaces.

    Azure Blob Storage get_object_cdn_url can't handle files with spaces.

    Summary

    I've been trying to generate a signed download link to a file with spaces in its name, let's call it file name.txt, and it has been failing to generate a token. While, If I use Azure's frontend, I can generate a token that works. I suspect the signature is calculated incorrectly.

    Detailed Information

    I had theorized that, azure don't urlencode the resource because it would be pointless if you are going to hash it. I've checked the official implementation and indeed they don't and use raw resource name when creating the signature. After that, I've created a subclass of AzureBlobsStorageDriver with get_object_cdn_url overloaded, and I've edited the string_to_sign's content with this:

    + "/blob/{}{}".format(self.key, unquote(object_path)),
    - "/blob/{}{}".format(self.key, object_path),
    

    And indeed it works now for any file I tried.

    My Environment: python==3.10.5 libcloud==3.6.0 operating system is official python:3.10 from docker

    opened by Cediddi 0
Owner
The Apache Software Foundation
The Apache Software Foundation
python-bigquery Apache-2python-bigquery (🥈34 · ⭐ 3.5K · 📈) - Google BigQuery API client library. Apache-2

Python Client for Google BigQuery Querying massive datasets can be time consuming and expensive without the right hardware and infrastructure. Google

Google APIs 550 Jan 1, 2023
Application which allows you to make PostgreSQL databases with Python

Automate PostgreSQL Databases with Python Application which allows you to make PostgreSQL databases with Python I used the psycopg2 library which is u

Marc-Alistair Coffi 0 Dec 31, 2021
Making it easy to query APIs via SQL

Shillelagh Shillelagh (ʃɪˈleɪlɪ) is an implementation of the Python DB API 2.0 based on SQLite (using the APSW library): from shillelagh.backends.apsw

Beto Dealmeida 207 Dec 30, 2022
Databank is an easy-to-use Python library for making raw SQL queries in a multi-threaded environment.

Databank Databank is an easy-to-use Python library for making raw SQL queries in a multi-threaded environment. No ORM, no frills. Thread-safe. Only ra

snapADDY GmbH 4 Apr 4, 2022
DataStax Python Driver for Apache Cassandra

DataStax Driver for Apache Cassandra A modern, feature-rich and highly-tunable Python client library for Apache Cassandra (2.1+) and DataStax Enterpri

DataStax 1.3k Dec 25, 2022
Python client for Apache Kafka

Kafka Python client Python client for the Apache Kafka distributed stream processing system. kafka-python is designed to function much like the offici

Dana Powers 5.1k Jan 8, 2023
PyRemoteSQL is a python SQL client that allows you to connect to your remote server with phpMyAdmin installed.

PyRemoteSQL Python MySQL remote client Basically this is a python SQL client that allows you to connect to your remote server with phpMyAdmin installe

ProbablyX 3 Nov 4, 2022
Easy-to-use data handling for SQL data stores with support for implicit table creation, bulk loading, and transactions.

dataset: databases for lazy people In short, dataset makes reading and writing data in databases as simple as reading and writing JSON files. Read the

Friedrich Lindenberg 4.2k Jan 2, 2023
Import entity definition document into SQLie3. Manage the entity. Also, create a "Create Table SQL file".

EntityDocumentMaker Version 1.00 After importing the entity definition (Excel file), store the data in sqlite3. エンティティ定義(Excelファイル)をインポートした後、データをsqlit

G-jon FujiYama 1 Jan 9, 2022
Google Cloud Client Library for Python

Google Cloud Python Client Python idiomatic clients for Google Cloud Platform services. Stability levels The development status classifier on PyPI ind

Google APIs 4.1k Jan 1, 2023
A library for python made by me,to make the use of MySQL easier and more pythonic

my_ezql A library for python made by me,to make the use of MySQL easier and more pythonic This library was made by Tony Hasson , a 25 year old student

null 3 Nov 19, 2021
MongoX is an async python ODM for MongoDB which is built on top Motor and Pydantic.

MongoX MongoX is an async python ODM (Object Document Mapper) for MongoDB which is built on top Motor and Pydantic. The main features include: Fully t

Amin Alaee 112 Dec 4, 2022
Pandas on AWS - Easy integration with Athena, Glue, Redshift, Timestream, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretManager, PostgreSQL, MySQL, SQLServer and S3 (Parquet, CSV, JSON and EXCEL).

AWS Data Wrangler Pandas on AWS Easy integration with Athena, Glue, Redshift, Timestream, QuickSight, Chime, CloudWatchLogs, DynamoDB, EMR, SecretMana

Amazon Web Services - Labs 3.3k Dec 31, 2022
A wrapper around asyncpg for use with sqlalchemy

asyncpgsa A python library wrapper around asyncpg for use with sqlalchemy Backwards incompatibility notice Since this library is still in pre 1.0 worl

Canopy 404 Dec 3, 2022
Python interface to Oracle Database conforming to the Python DB API 2.0 specification.

cx_Oracle version 8.2 (Development) cx_Oracle is a Python extension module that enables access to Oracle Database. It conforms to the Python database

Oracle 841 Dec 21, 2022
A tutorial designed to introduce you to SQlite 3 database using python

SQLite3-python-tutorial A tutorial designed to introduce you to SQlite 3 database using python What is SQLite? SQLite is an in-process library that im

null 0 Dec 28, 2021
PubMed Mapper: A Python library that map PubMed XML to Python object

pubmed-mapper: A Python Library that map PubMed XML to Python object 中文文档 1. Philosophy view UML Programmatically access PubMed article is a common ta

灵魂工具人 33 Dec 8, 2022
Python version of the TerminusDB client - for TerminusDB API and WOQLpy

TerminusDB Client Python Development status ⚙️ Python Package status ?? Python version of the TerminusDB client - for TerminusDB API and WOQLpy Requir

TerminusDB 66 Dec 2, 2022
Google Sheets Python API v4

pygsheets - Google Spreadsheets Python API v4 A simple, intuitive library for google sheets which gets your work done. Features: Open, create, delete

Nithin Murali 1.4k Dec 31, 2022