ARA Records Ansible and makes it easier to understand and troubleshoot.

This adds a new task "index" page much like the one for playbooks and hosts. It is modelled after /api/v1/tasks.

It's up for testing purposes on demo.recordsansible.org/tasks right now and looks like this:

Hi @dmsimard,

I had another idea how to solve the "distinct hosts problem" (compare discussion here #218), with this WIP PR I want to share my idea and open the discussion:

The idea is to create a new model DistinctHost with basically only two fields:

• name: the hostname, which will be the primary key of the database table
• latest_host: which is a ForeignKey to the Host model

Thus in this model we have alway a list with all distinct hosts and directly the relation to the latest Host object.

The trick now is to update the DistinctHost model if a new Host model entry will be created. We could do that already by overwriting the save method of the Host model, but I think that's not a best practice. And because in our case we only create Host entries via the API, I implemented a perform_create method on the HostViewSet, which updates (update_or_create) the DistinctHost model via an DistinctHostSerializer.

For development and testing purposes I've also already added a API endpoint /api/v1/distincthosts with a ListSerializer and the enpoints /api/v1/distincthosts/HOSTNAME with a DetailedSerializer. Surely there could be still some improvements, but for first tests it should be enough.

I haven't yet tested the performance in scale, but I do not see a reason why it should be slow.

The advantage of my approach here, is that we do not touch existing model, but only introduce a new one. But with that we could now implement all the things we thought about in #218. Furthermore I see also an advantage in having an API endpoint with which you can directly check the last result for a host (/api/v1/distincthosts/HOSTNAME).

Ansible is moving forward with an approach called Ansible collections, see "The Future of Ansible Content Delivery".

ARA has a lot of Ansible content and it might be worthwhile to consider packaging everything in an Ansible collection:

• Action and callback plugins
• Roles
• Playbooks
• Integration tests

Docs:

• User guide: https://github.com/ansible/ansible/blob/devel/docs/docsite/rst/user_guide/collections_using.rst
• Dev guide: https://github.com/ansible/ansible/blob/devel/docs/docsite/rst/dev_guide/developing_collections.rst

Let's use this issue to discuss how we might do this.

I updated to ara 1.5.6 hoping that it will fix it (see https://github.com/ansible-community/ara/issues/211), but exceptions remain:

[WARNING]: Failure using method (v2_playbook_on_play_start) in callback plugin (<ansible.plugins.callback.ara_default.CallbackModule object at 0x7f9ff7d6d3a0>): 'id'
[WARNING]: Failure using method (v2_playbook_on_task_start) in callback plugin (<ansible.plugins.callback.ara_default.CallbackModule object at 0x7f9ff7d6d3a0>): 'id'

[WARNING]: Failure using method (v2_runner_on_ok) in callback plugin (<ansible.plugins.callback.ara_default.CallbackModule object at 0x7f9ff7d6d3a0>): 'NoneType' object is not subscriptable

[WARNING]: Failure using method (v2_playbook_on_handler_task_start) in callback plugin (<ansible.plugins.callback.ara_default.CallbackModule object at 0x7f9ff7d6d3a0>): 'id'
[WARNING]: Failure using method (v2_runner_on_skipped) in callback plugin (<ansible.plugins.callback.ara_default.CallbackModule object at 0x7f9ff7d6d3a0>): 'NoneType' object is not subscriptable

[WARNING]: Failure using method (v2_playbook_on_stats) in callback plugin (<ansible.plugins.callback.ara_default.CallbackModule object at 0x7f9ff7d6d3a0>): 'id'

Versions:

Ansible Version: 2.10.6
ARA Version: 1.5.6

Server side logs:

bal. 19 13:19:56 localhost gunicorn[511750]: 2021-04-19 13:19:56,084 WARNING django.request: Bad Request: /api/v1/plays

In use cases where playbooks are run against localhost, it might be useful to record results under the fqdn of the host instead in order to differentiate task results belonging to different hosts.

TODO:

• integration tests
• docs

Hi @dmsimard,

just checked your last work on the UI (PRs #226 and #227):

As I already told you a few times, the new theme is great, the dark one, but also the bright one. Here just some feedback and possible improvements from my side:

• for me it is really hard to read the labels in the dark theme

• due to the sorting option, the Date and Duration column is not aligned in the playbook index page:

• with https://github.com/ansible-community/ara/commit/90550b32393d16a758457ef776ac2fa9e09cc6bf you made the playbook reports card scrollable and constrained the height, which is great! Becasue the three cards are on the same row and the hosts cards is expanded by default, it would also make sense to expand the Files and Records cards per default as well (currently they are collapsed). At least if the screen is wide enough.

• the heading of the file in the file page (e.g. https://dev.demo.recordsansible.org/files/201724.html) is really huge:

• the heading of the results in loops is also really huge (/results/XXXXX.html)

• Maybe this is just because I was used to it, but I would put the Status column at the front of the row in the playbook index page. For me the first column is the eye catcher, and the most important thing is if the playbook was sucessfull or not. And I think the CLI column is not that important, thus I woul put it on the end, just before the Labels columen: (see hille721/ara@6af62b3a551d0bf0ceb3b9ae3f9e5df0c04c6e2c)

What is the issue ?

When Running ARA inside AWX i do get exceptions on some methods. As a result the playbook will never show finished in ARA and the data is incomplete. When running it from my workstation there is no problem.

Exceptions:

• Failure using method (v2_playbook_on_play_start) in callback plugin (<ansible.plugins.callback.ara_default.CallbackModule object at 0x7f5a4d94fda0>): 'id'
• Failure using method (v2_runner_on_skipped) in callback plugin (<ansible.plugins.callback.ara_default.CallbackModule object at 0x7f5a4d94fda0>): 'NoneType' object is not subscriptable
• Failure using method (v2_playbook_on_stats) in callback plugin (<ansible.plugins.callback.ara_default.CallbackModule object at 0x7f5a4d94fda0>): 'id'

Versions:

• Ansible Version: 2.9.18
• AWX Version 17.0.1
• ARA Version: 1.5.4

Server side Logs at the end of the playbook:

[ara-7699dc477c-s9s9t ara] 10.200.212.83 - admin [04/Mar/2021:10:08:43 +0000] "PATCH /api/v1/plays/3 HTTP/1.1" 200 328 "-" "ara-http-client_1.5.4" 
[ara-7699dc477c-s9s9t ara] Bad Request: /api/v1/plays 
[ara-7699dc477c-s9s9t ara] 2021-03-04 10:08:44,077 WARNING django.request: Bad Request: /api/v1/plays 
[ara-7699dc477c-s9s9t ara] 10.200.212.83 - admin [04/Mar/2021:10:08:44 +0000] "POST /api/v1/plays HTTP/1.1" 400 34 "-" "ara-http-client_1.5.4" 

Related Client log message:

WARNING]: Failure using method (v2_playbook_on_stats) in callback plugin
(<ansible.plugins.callback.ara_default.CallbackModule object at
0x7f5a4d94fda0>): 'id'
Callback Exception: 
  File "/opt/custom-venvs/xx/lib/python3.6/site-packages/ansible/executor/task_queue_manager.py", line 381, in send_callback
    method(*new_args, **kwargs)
   File "/opt/custom-venvs/xx/lib/python3.6/site-packages/ara/plugins/callback/ara_default.py", line 405, in v2_playbook_on_stats
    self._end_play()
   File "/opt/custom-venvs/xx/lib/python3.6/site-packages/ara/plugins/callback/ara_default.py", line 430, in _end_play
    "/api/v1/plays/%s" % self.play["id"],    

The callback will retrieve files that are loaded by Ansible throughout the execution of the playbook. This could contain files that users might not want to save, either because they are sensitive or are simply not relevant.

The implementation would be somewhat similar to that we already do with ignored_arguments and ignored_facts for ignoring specified playbook parameters and host facts.

What is the idea ?

It would be greate to have a possibilty to search for a host in the web ui of the api server:

The result shoud be then all playbook runs with this host.

What component is this about ?

ara-server, mainly rest-api

What is your ARA installation like ?

Found in version 1.3.2 (from debian package) but same code is in master

What is happening ?

When EXTERNAL_AUTH is enabled, only admin and healthcheck working properly. /api and index return 401

What should be happening ?

Everything should be working same as it internal auth is used.

Main problem is in server.settings because rest_framework use BasicAuth even if EXTERNAL_AUTH is set. This patch should fix issue

diff --git a/ara/server/settings.py b/ara/server/settings.py
index 252fa98..b4691d1 100644
--- a/ara/server/settings.py
+++ b/ara/server/settings.py
@@ -214,6 +214,12 @@ ROOT_URLCONF = "ara.server.urls"
 APPEND_SLASH = False
 
 PAGE_SIZE = settings.get("PAGE_SIZE", 100)
+
+if EXTERNAL_AUTH:
+    REST_FRAMEWORK_AUTH = ('rest_framework.authentication.RemoteUserAuthentication',)
+else:
+    REST_FRAMEWORK_AUTH = ('rest_framework.authentication.BasixAuthentication',)
+
 REST_FRAMEWORK = {
     "DEFAULT_PAGINATION_CLASS": "rest_framework.pagination.LimitOffsetPagination",
     "PAGE_SIZE": PAGE_SIZE,
@@ -227,7 +233,7 @@ REST_FRAMEWORK = {
         "rest_framework.parsers.FormParser",
         "rest_framework.parsers.MultiPartParser",
     ),
-    "DEFAULT_AUTHENTICATION_CLASSES": ("rest_framework.authentication.BasicAuthentication",),
+    "DEFAULT_AUTHENTICATION_CLASSES": REST_FRAMEWORK_AUTH ,
     "DEFAULT_PERMISSION_CLASSES": ("ara.api.auth.APIAccessPermission",),
     "TEST_REQUEST_DEFAULT_FORMAT": "json",
     "UNICODE_JSON": False,

NOTE: sorry If this is not a proper way to post patch. There are no pull request so I have no idea where to put this.

What component is this about ?

Ara API server

What is your ARA installation like ? I have installed Ara1.1 , with pip install inside a virtual environment Ansible 2.9,psql (9.2.24, server 11.5)

What is happening ? Ara UI is loading very slow , it gives 504 timed out error, Some time I get 'too many opened files' other times I dont see any error in the nohup output and server side its giving 200 response, but in UI it is timing out. I am attached the db data details

I had this issue with sqlite, thought its db issue. Now I have changed to postgresql and migrated the old data to it. again I am getting the same error. After restart, it is better for sometime and the problem occurs again in few minutes. our setup is in aws environment , ara in one server and postgres in rds but having it in same setup also had this issue when the playbook is more than 300. this is server nohup output at that time of the issue.

What should be happening ? UI should load irrespective of db load.

What is the issue ?

Using ara under Mac OS X, the hostname that python picks up on my machine is 1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa which isn't particularly useful when the hostname and hostname -f commands both return a proper hostname like laptop.example.org.

We rely on python's socket.getfqdn() and socket.gethostname() to retrieve the hostname: https://github.com/ansible-community/ara/blob/196e256589893745c9f732c84f8edb7662997846/ara/plugins/callback/ara_default.py#L829-L833

On OS X (at least on my machine, I'm an OS X noob), getfqdn returns the ipv6 arpa mentioned previously and gethostname returns the actual hostname.

What should be happening ?

There's settings that determine which format is used:

In this particular case, hostname would probably be a better default than fqdn. Is this generally true for all distros ?

Changing it could result in playbooks being associated to a "new" controller if the fqdn and hostname are not the same.

Happy to hear what you think.

What is the issue ?

When using ARA 1.6.0 with a HTTPS API Server the callback plugin will fail due to a certificate error. It seems like it is not checking the default cert bundle on they system (Fedora 37 in my case) as the following fixes the error

[ara]
api_client = http
api_server = https://ara.example.com
api_ca = /etc/pki/tls/cert.pem

Without api_ca the connection fails with

[WARNING]: Failure using method (v2_playbook_on_start) in callback plugin
(<ansible.plugins.callback.ara_default.CallbackModule object at 0x7f4b47a812d0>):
HTTPSConnectionPool(host='ara.nmsu.edu', port=443): Max retries exceeded with url:
/api/v1/playbooks (Caused by SSLError(SSLCertVerificationError(1, '[SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer
certificate (_ssl.c:992)')))

PLAY [Test Ansible Connection] ************************************************************
[WARNING]: Failure using method (v2_playbook_on_play_start) in callback plugin
(<ansible.plugins.callback.ara_default.CallbackModule object at 0x7f4b47a812d0>):
'NoneType' object is not subscriptable

TASK [connection test] ********************************************************************
[WARNING]: Failure using method (v2_playbook_on_task_start) in callback plugin
(<ansible.plugins.callback.ara_default.CallbackModule object at 0x7f4b47a812d0>):
'NoneType' object is not subscriptable
ok: [hpc-wiki-p]

TASK [stop ssh client persistant connection] **********************************************

PLAY RECAP ********************************************************************************
hpc-wiki-p                 : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   

[WARNING]: Failure using method (v2_playbook_on_stats) in callback plugin
(<ansible.plugins.callback.ara_default.CallbackModule object at 0x7f4b47a812d0>):
'NoneType' object is not subscriptable

What should be happening ?

The default ca bundle trusted by the system should be used by default when api_ca is not specified

Software Versions

Ansible = 7.1.0 (Core = 2.14) OS = Fedora 37 Python = 3.11 ARA = 1.6.0 Install Source = pip

During an automic update of one ara client I recognized a compatibility issue between ara client version 1.6.0 and ara server 1.5.8. The playbook get's created in ara, but updating fails:

...
[WARNING]: Failure using method (v2_playbook_on_task_start) in callback plugin (<ansible.plugins.callback.ara_default.CallbackModule object at 0x7f3cafeb1610>): 'uuid'
Callback Exception: 
  File "/home/hille/gitrepos/plminfra_ansible/.venv/lib/python3.9/site-packages/ansible/executor/task_queue_manager.py", line 450, in send_callback
    method(*new_args, **kwargs)
   File "/home/hille/gitrepos/plminfra_ansible/.venv/lib/python3.9/site-packages/ara/plugins/callback/ara_default.py", line 504, in v2_playbook_on_task_start
    self._end_task()
   File "/home/hille/gitrepos/plminfra_ansible/.venv/lib/python3.9/site-packages/ara/plugins/callback/ara_default.py", line 602, in _end_task
    task_uuid = self.task["uuid"]
...

[WARNING]: Failure using method (v2_runner_on_ok) in callback plugin (<ansible.plugins.callback.ara_default.CallbackModule object at 0x7f3cafeb1610>): 'NoneType' object has no attribute 'submit'
Callback Exception: 
  File "/home/hille/gitrepos/plminfra_ansible/.venv/lib/python3.9/site-packages/ansible/executor/task_queue_manager.py", line 450, in send_callback
    method(*new_args, **kwargs)
   File "/home/hille/gitrepos/plminfra_ansible/.venv/lib/python3.9/site-packages/ara/plugins/callback/ara_default.py", line 534, in v2_runner_on_ok
    self._submit_thread("task", self._load_result, result, "ok", **kwargs)
   File "/home/hille/gitrepos/plminfra_ansible/.venv/lib/python3.9/site-packages/ara/plugins/callback/ara_default.py", line 359, in _submit_thread
    threads.submit(func, *args, **kwargs)
...

[WARNING]: Failure using method (v2_playbook_on_stats) in callback plugin (<ansible.plugins.callback.ara_default.CallbackModule object at 0x7f3cafeb1610>): 'uuid'
Callback Exception: 
  File "/home/hille/gitrepos/plminfra_ansible/.venv/lib/python3.9/site-packages/ansible/executor/task_queue_manager.py", line 450, in send_callback
    method(*new_args, **kwargs)
   File "/home/hille/gitrepos/plminfra_ansible/.venv/lib/python3.9/site-packages/ara/plugins/callback/ara_default.py", line 586, in v2_playbook_on_stats
    self._end_task()
   File "/home/hille/gitrepos/plminfra_ansible/.venv/lib/python3.9/site-packages/ara/plugins/callback/ara_default.py", line 602, in _end_task
    task_uuid = self.task["uuid"]

In ara the playbook will be then shown forever as running.

Good thing is, running ara server 1.6.0 and ara client 1.5.8 is working without any problems (of course some things introduced with 1.6.0 are then n/a in the GUI:

This issues should be more a tip for everyone, that the upgrade path to 1.6.0 should be:

1. upgrade server
2. upgrade all your clients

This ensures that ara is still working also in a transition phase.

@dmsimard : maybe you could also announce this somewhere.

What is the idea ?

Filing this under nice to have because ara records host facts when Ansible gathers them so they are available but we are not doing much right now other than pretty printing them:

It's still better than nothing but there could -- perhaps -- be some sort of compact summary of the node if the facts are available. You can see an example of all the facts available on the demo: https://demo.recordsansible.org/hosts/11031.html

For example:

• We have a bunch of useful variables like ansible_distribution(_version), ansible_kernel, ansible_processor, ansible_python, etc.
• There's network information available with facts like ansible_default_ipv4 and for specific network interfaces (like ansible_br0)
• With ansible_memory_mb we know the amount of memory, how much is used, how much is free
• With ansible_mount we know about block devices, partitions, filesystems and disk usage
• With ansible_loadavg (starting in ansible-core 2.14) we know, well, the load avg.

The facts are not always gathered and even if they are, the ones we would use could be filtered out, so the implementation should be tolerant to missing facts or none at all.

It's just a nice to have when facts are available ? ¯\(ツ)/¯

What is the idea ?

I do not have a mock up or have a strong opinion on what it should look like (yet?) but it would be great to have some sort of breadcrumbs or navigation at the top of most pages. There's bootstrap documentation on breadcrumbs but I'm open to ideas or suggestions.

The relationship between objects is explained in the docs:

With that in mind, if we had "breadcrumbs", some examples might be:

# https://demo.recordsansible.org/results/864718.html
index > playbook > play > task > result
# https://demo.recordsansible.org/files/339706.html
index > playbook > file
# https://demo.recordsansible.org/hosts/11043.html
hosts > host facts > task results

Another set of breadcrumbs (or table of contents?) would be nice to display what content is available that may be much further down the page. Maybe a list-group ?

For example, when viewing a host, you might see links to the playbook summary, the host facts and the task results:

This would be particularly useful in cases where there is a lot of data on a single page so the user knows there are other things "hidden" underneath. The example above had a few facts but full fact gathering pushes the task results down.

What is the idea?

Hi, I am working with ARA as part of my work optimization and I consider very helpful beeing capable of recording some additional data on ARA plays. One kind of value that I record (using 'ara_record' module) is a url, that is why I set in the ansible task the type 'url' but I expect ARA to record it as a hyperlink so I can click on it and go directly to the navigator. Instead of it, the value keeps beeing recorded as text.

¿Is there any chance to achieve this feature? It would be very helpful to record hyperlinks.

Thank you so much