This is a very bare-bone implementation for the self-hosted Postal platform.

About Postal:

Postal is a complete and fully featured mail server for use by websites & web servers. Think Sendgrid, Mailgun or Postmark but open source and ready for you to run on your own servers.

We use this draft implementation since a few days to handle emails for our karrot.world web app. Before we used SparkPost via anymail. I had good experiences with anymail so far, that's why I'm keen on using it. I'll improve this implementation during the next weeks as I find time. I'm happy about early feedback!

To do:

• [x] figure out how to do webhook authentication with Postal
• [ ] support track_clicks and track_opens
• [x] add backend tests
• [x] add webhook tests
• [x] add docs
• [x] integration tests?

I just seemlessly migrated from djrill to django-anymail. Thanks for making it so easy!

Now I'm looking to change ESP from Mandrill, and have found SparkPost to offer a very generous number of free emails, and a very similar user interface to Mandrill. Not only that, but they have a tool to migrate templates.

It would be great if it were supported by django-anymail.

Thanks.

We are using Anymail 8.5, Django 3.2.12, Python 3.9.1 and have for a while been getting MultiValueDictKeyError ( in request.FILES[att_id] ) on some inbound Mailgun emails here https://github.com/anymail/django-anymail/blob/main/anymail/webhooks/mailgun.py#L406

attachments = [
    AnymailInboundMessage.construct_attachment_from_uploaded_file(
        request.FILES[att_id], content_id=att_cids.get(att_id, None))
    for att_id in att_ids
]

It seems request.POST[”attachment-count”] doesn’t match number of files in request.FILES and attachment-count doesn’t match number of attachment-%d. I don’t know if this is Mailgun removing some files or why, but maybe we could make Anymail handle it. We are seeing quite a few inbound mails not coming through at all because of this mismatch in counts. After all it would be better to have them come through with missing attachments rather than not coming at all, perhaps with an added flag of missing attachement so each implementation can handle potentially missing attachments.

Below is an example of the relevant section of the request.POST content. Attachment-count (5), content-id-map (3) and actual attachment keys (2) don’t match.

{
    'attachment-1': ['”BINARY_CONTENT”], 
    'attachment-2': [”BINARY_CONTENT”], 
    'attachment-count': ['5'], 
    'content-id-map': ['{"":"attachment-1","":"attachment-2","":"attachment-3"}'], 
}

• Anymail version Latest, 6.0.1

• ESP (Mailgun, SendGrid, etc.) Sendgrid

• Your ANYMAIL settings (change secrets to "redacted") Default

• Versions of Django, requests, python django==1.11.20 requests==2.20.0 python==3.6.8

• Exact error message and/or stack trace I'm sending an icalendar (ICS) file as attachment. The file's content is text (I don't think the exact content is relevant here, just note that there are some non-ascii chars around, like ü, and ä):

BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//flatfox//flatfox.ch//
METHOD:REQUEST
BEGIN:VEVENT
SUMMARY:Viewing at Schreinerstrasse 64\, 8004 Zürich
DTSTART;VALUE=DATE-TIME:20190526T150000Z
DTEND;VALUE=DATE-TIME:20190526T153000Z
DTSTAMP;VALUE=DATE-TIME:20190523T121031Z
UID:[email protected]
SEQUENCE:9
ATTENDEE;CN="Bernhard Mäder";CUTYPE=INDIVIDUAL;PARTSTAT=ACCEPTED;ROLE=REQ
 -PARTICIPANT;RSVP=FALSE;X-NUM-GUESTS=0:MAILTO:<some_email>@flatfox.ch
DESCRIPTION:Address: Schreinerstrasse 64\, 8004 Zürich\nContact person: B
 ernhard Mäder\nWhen: Sun\, 05/26/2019\, 5 p.m.\nAttendees: Max Muster - 2
 738492379847
LOCATION:Schreinerstrasse 64\, 8004 Zürich
ORGANIZER;CN=flatfox:MAILTO:<no_reply>@flatfox.ch
PRIORITY:5
STATUS:CONFIRMED
BEGIN:VALARM
ACTION:DISPLAY
DESCRIPTION:Viewing at Schreinerstrasse 64\, 8004 Zürich
TRIGGER:-PT30M
END:VALARM
END:VEVENT
END:VCALENDAR

The resulting payload (as received by the email client, after the sendgrid roundtrip) is as follows:

--cc91f1040f2ddb86e28e23abd2298e8686508c1291386a1d824cb54aebce
Content-Disposition: attachment; filename="termin.ics" Content-Transfer-Encoding: base64 Content-Type: text/calendar; name=termin.ics; charset=iso-8859-1
QkVHSU46VkNBTEVOREFSDQpWRVJTSU9OOjIuMA0KUFJPRElEOi0vL2ZsYXRmb3gvL2ZsYXRmb3gu
Y2gvLw0KTUVUSE9EOlJFUVVFU1QNCkJFR0lOOlZFVkVOVA0KU1VNTUFSWTpCZXNpY2h0aWd1bmcg
ZvxyIFNjaHJlaW5lcnN0cmFzc2UgNjRcLCA4MDA0IFr8cmljaA0KRFRTVEFSVDtWQUxVRT1EQVRF
LVRJTUU6MjAxOTA1MjZUMTUwMDAwWg0KRFRFTkQ7VkFMVUU9REFURS1USU1FOjIwMTkwNTI2VDE1
MzAwMFoNCkRUU1RBTVA7VkFMVUU9REFURS1USU1FOjIwMTkwNTIzVDEyMzUzNFoNClVJRDo2Mzhi
YjBjYS1mMjhhLTQ4YWItYWVkOS04ODYxYmY1ZTg3NGQuMTY2OTg2QGZsYXRmb3guY2gNClNFUVVF
TkNFOjENCkFUVEVOREVFO0NOPSJNYXggTXVzdGVyIjtDVVRZUEU9SU5ESVZJRFVBTDtQQVJUU1RB
VD1BQ0NFUFRFRDtST0xFPVJFUS1QQVJUDQogSUNJUEFOVDtSU1ZQPUZBTFNFO1gtTlVNLUdVRVNU
Uz0wOk1BSUxUTzpiZXJuaGFyZC5tYWVkZXJAb3V0bG9vay5jb20NCkRFU0NSSVBUSU9OOkFkcmVz
c2U6IFNjaHJlaW5lcnN0cmFzc2UgNjRcLCA4MDA0IFr8cmljaFxuS29udGFrdHBlcnNvbjogQmUN
CiBybmhhcmQgTeRkZXJcblplaXQ6IFNvXCwgMjYuMDUuMjAxOVwsIDE3OjAwDQpMT0NBVElPTjpT
Y2hyZWluZXJzdHJhc3NlIDY0XCwgODAwNCBa/HJpY2gNCk9SR0FOSVpFUjtDTj1mbGF0Zm94Ok1B
SUxUTzpjYWxlbmRhckBmbGF0Zm94LmNoDQpQUklPUklUWTo1DQpTVEFUVVM6Q09ORklSTUVEDQpC
RUdJTjpWQUxBUk0NCkFDVElPTjpESVNQTEFZDQpERVNDUklQVElPTjpCZXNpY2h0aWd1bmcgZvxy
IFNjaHJlaW5lcnN0cmFzc2UgNjRcLCA4MDA0IFr8cmljaA0KVFJJR0dFUjotUFQzME0NCkVORDpW
QUxBUk0NCkVORDpWRVZFTlQNCkVORDpWQ0FMRU5EQVINCg==
--cc91f1040f2ddb86e28e23abd2298e8686508c1291386a1d824cb54aebce--

The problem with this:

• The payload is encoded with UTF-8, then converted to base64
• The charset states iso-8859-1
• Some mail client (outlook.com in particular) will now use the charset to decode the payload, which renders some characters wrongly. This is per spec, where the ICS's encoding may be deduced from the container's.

So, I'd like the charset to be set to utf-8.

Also interesting, but not really important: gmail, thunderbird and a few other clients will (incorrectly) use UTF-8, as this is the default encoding for icalendar files.

• Any other relevant code and settings (e.g., for problems sending, your code that sends the message) None

Thank you for maintaining Anymail, it is really great so far.

I am trying to receive a webhook from Mailgun but I get an error.

"Missing or invalid basic auth in Anymail Mailgun webhook"

Anymail is configured correctly, and I have been sending email through Mailgun for about a week.

ANYMAIL = {
    "MAILGUN_API_KEY": Env.value("MAILGUN_API_KEY"),
    "MAILGUN_SENDER_DOMAIN": Env.value("MAILGUN_SENDER_DOMAIN"),
    "WEBHOOK_AUTHORIZATION": Env.value("ANYMAIL_WEBHOOK_AUTHORIZATION"),
}

Today I added the "WEBHOOK_AUTHORIZATION" setting. Currently I am testing this feature on our dev environment. Therefore, the Mailgun sender domain is my Mailgun sandbox domain.

I set the Mailgun "Unsubscribes" web hook to the url defined in the Anymail documentation.

https://secret1:[email protected]/anymail/mailgun/tracking/

And in my urls.py

url(
    r'^anymail/',
    include('anymail.urls')
),

When I receive an email and click the unsubscribe link, a POST is made to my server, but I get the error listed above.

For what it's worth, I am hosting my site on Webfaction using Apache.

I found a nearly identical issue on Stackoverflow, but no answer provided. https://stackoverflow.com/questions/38620608/django-anymail-mailgun-webhook-no-http-authorization-in-meta

dear, Im getting Mandrill webhook called with incorrect signature error with a Bad Request response towards mandrill.

Sentry logs say about the err location:

filename	
'base.py'
lineno	
216
pathname	
'/home/onlinegv/eggs/Django-1.9.9-py2.7.egg/django/core/handlers/base.py'

I triple checked my webhook and api key. my total settings look like this:

        ANYMAIL =  {
            'MANDRILL_API_KEY': get_config_or_none(
                config,'Email', 'mandrill_api_key', 'SOMEKEY-A'),
            'WEBHOOK_AUTHORIZATION':'SOMEAUTH:SOMEPW',
            'MANDRILL_WEBHOOK_KEY': get_config_or_none(
                config, 'Email', 'mandrill_webhook_key', 'SOMEKEY'),
        }

what else should I check to investigate the failure?

I'm currently evaluating django-anymail (with mailgun) which I will be using inbound mail with and see on your roadmap that you intend to add inbound mail support. Did you have a structure/plan in mind for doing this, before I do a quick specific implementation similar to the mailgun docs, which probably won't be suitable for django-anymail?

The docs claim that

Mailgun does not offer ESP stored templates, so Anymail’s template_id message attribute is not supported with the Mailgun backend.

but that does not appear to be true: https://documentation.mailgun.com/en/latest/api-templates.html

Am I mistaking either Anymail's or Mailgun's docs?

ESP: SendGrid django-anymail: 2.2 Python: 2.7 Django: 1.11.10

Just after we make the msg = AnymailMessage(**kwargs) call we store the data from msg.anymail_status in our database for tracking. The issue we're seeing is that the SendGrid webhook will often times return POST data with a message_id and esp_event['smtp-id] that do not match the values Anymail generated for us.

As an example: We send a message and receive an smtp-id of <[email protected]> in the msg.anymail_status. When the webhook communicates updates to us we find that the message_id for the same message is <[email protected]> which means that we are unable to identify which message this belongs to.

Here's the code that calls AnymailMessage:

text_message = render_to_string(templates['text'], context)
html_message = render_to_string(templates['html'], context)
msg = AnymailMessage(
    subject=templates['subject'],
    body=text_message,
    to=[user.email],
)
msg.attach_alternative(html_message, "text/html")
msg.send()

# store the anymail status data to our database for tracking via the webhook
Email().create_from_anymail(msg.anymail_status, Email.INVITATION)

The Email() model holds the create_from_anymail(anymail_status, event) method where event is an optional custom tracker for the event triggering the email send (unrelated to anymail):

@classmethod
def create_from_anymail(cls, anymail_status, event=None):
    for recipient in anymail_status.recipients:
        kwargs = {
            'to_email': recipient,
            'status': anymail_status.recipients[recipient].status,
            'message_id': anymail_status.recipients[recipient].message_id,
            'response': anymail_status.esp_response
        }

        if event:
            kwargs['event'] = event

        email = Email(**kwargs)
        email.save()

From what I can gather from the SendGrid webhook documentation the smtp-id should be returned for every webhook POST and is a unique ID attached to the message by the originating system.

I've also contacted SendGrid support but wanted to present the issue here in case anyone has seen something similar and may have ideas on how to get a consistent identifier from our system into SendGrid.

HI! We have recently changed our ESP from Sparkpost to Mailgun. Using this librairy has saved us a lot of time. We thank you for that!

Everything works well except for one annoying detail: The from email gets surrounded by quotes when containing a special character like 'é' or 'è'. This was not an issue when using Sparkpost.

Here's an example of what I mean:

def send_welcome_email(from_email, recipients):
    html_content = get_template("welcome.html")
    message = EmailMultiAlternatives(
        subject="Welcome to MyApp",
        body="Welcome!",
        from_email=from_email,
        to=recipients,
    )
    message.attach_alternative(html_content, "text/html")
    message.send()

Now if we use this method like this:

send_welcome_email("Felix (via MyApp) <[email protected]>", ["[email protected]"]) 

I get the expected result (In Gmail, when I click "Show original" I can see the following)

From: "Felix (via MyApp)" <[email protected]>

But, if I use the method like this:

send_welcome_email("Félix (via MyApp) <[email protected]>", ["[email protected]"])  # Notice the 'é' instead of the 'e'

I get this result:

From: "\"Félix (via MyApp)\"" <[email protected]>

When we were using Sparkpost, this was not a problem (we got the expected result with the 'é'). It looks quite ugly and unprofessional in the recipients' mailboxes, and since our application is aimed at french users and many french names have special characters like 'é' or 'è' in them, this is a big problem for us.

Hope someone can help :) Thank you!

• Anymail version: 8.4.0
• ESP: Mailgun (previously Sparkpost)
• Django 3.2, python 3.9.6

Reporting an error? It's helpful to know:

• Anymail version: 5.0
• ESP : Mailgun
• Versions of Django 2.1.7
• Exact error message and/or stack trace

AnymailRequestsAPIError at /emails/ Invalid JSON in Mailgun API response Sending a message to [email protected] from [email protected] Mailgun API response 200: OK Mailgun Magnificent API

Exception Value: Invalid JSON in Mailgun API response Sending a message to [email protected] from [email protected] Mailgun API response 200: OK Mailgun Magnificent API

Anymail is throwing an exception, but mailgun is sending 200

Hi,

We really appreciate guys if you could integrate mailersend ESP (https://mailersend.com), and Sendpulse ESP (https://sendpulse.com/)

it's a reaaaaly serious player in the ESP space, with 12,000 to 15,000 emails/month Send for free

Thank you

Just a suggestion: would it be possible to add a "reduced" DEBUG_API_REQUESTS implementation to log only basic information about the requests and responses (eg. status code, possibly errors in the response; a subset of fields in the request) to allow use outside dev environments? I'm asking because we had an impossible to track intermittent problem with some notifications (ESP is Postmark) - all our logs confirmed that the email was sent, but nothing was apparently received by Postmark, and Anymail showed no error.

It will be easier to contribute if the style preferences of the project were enforced automatically. pre-commit is a tool that supports this. I've set it up to apply black, isort and doc8 to standardize the coding style.

Obviously, this is a large change and I don't have a ton of involvement with the project historically. However, these tools have made my life easier on other projects, especially those that I am reviewing pull requests for.

Edit: If this is too much to take on, that's understandable and I won't be offended.

It was recommended in the docs to use the default Django EmailBackend for SMTP. However, by doing this, we are losing the advantages of the pre- and post-send signals.

The use-case that we have right now is we are using a mail catcher on the local setup (which is SMTP) and then switches our backend to Mailjet on production.

The issue is that we had to write our own SMTP backend to get the same behavior as one of the Anymail-supported backends' pre- and post-send.

Anymail has the test and console backends but is missing the SMTP backend.

Hi, this is just a heads-up. I've noticed that AWS released SESV2 API which is a bit different from V1 implemented in anymail.

There is a change that "Raw" email method is changed to parameter of SendEmail

See affected methods:

V1 SendRawEmail SendBulkTemplatedEmail V2 SendBulkEmail v2 SendEmail v2

This also results in IAM permissions changes

V1

"ses:SendRawEmail",
"ses:SendBulkTemplatedEmail",

V2

"ses:SendEmail",
"ses:SendBulkEmail"

PS. Feel free to close this issue if you do not plan to update because it may be unlikely that AWS is going to deprecate standard SES API any time soon.