From my point of view, it would be enough if IronPython 3.x only supports .net framework version equal-greater than 4.5.1. This may make the build and test-process more easy.

Here is my exploration around the re-implementation of the buffer protocol (#21). I will try to make incremental changes that will keep IronPython working (so all tests should be passing).

Each step is a proposal of an idea that can be discussed and adjusted, so smaller steps are preferable, rather a large dive that goes nowhere.

The PR is writable to the maintainers, so I welcome commits on top in a collaborative effort.

Things left to do for the release:

• [x] Release DLR 1.3.2 and bump to nuspec
• [x] Finish and merge https://github.com/IronLanguages/ironpython3/pull/1417 and https://github.com/IronLanguages/ironpython3/pull/1420
• [x] Update installer?
• [x] Document breaking changes related to https://github.com/IronLanguages/ironpython3/issues/52
• [ ] Prepare release notes (special mentions: f-strings, venv, int/long unification, anything else?)
• [x] Prepare website update with release info
• [ ] Testing
• [ ] Other stuff?

@BCSharp Any potential PRs you'd like to squeeze in before a release? Can you think of anything else?

When starting ipy on MacOS I get LookupError: unknown encoding: cp0.

I compiled IronPython for Mac from source as I did not find any binary releases for IronPython (homebrew?)

➜  ironpython3 git:(35cc2274) ✗ pwsh make.ps1 -frameworks net6.0 debug

➜  ironpython3 git:(35cc2274) ✗ ./bin/Debug/net6.0/ipy
Traceback (most recent call last):
  File "<string>", line 1567, in load_module
  File "<string>", line 596, in _load_module_shim
  File "<string>", line 1210, in load
  File "<string>", line 1190, in _load_unlocked
  File "<string>", line 1119, in _exec
  File "<string>", line 1461, in exec_module
  File "<string>", line 539, in _check_name_wrapper
  File "<string>", line 321, in _call_with_frames_removed
  File "/Users/sascha/Dev/ironpython3/bin/Debug/net6.0/../../../Src/StdLib/Lib/site.py", line 564, in <module>
  File "/Users/sascha/Dev/ironpython3/bin/Debug/net6.0/../../../Src/StdLib/Lib/site.py", line 550, in main
  File "/Users/sascha/Dev/ironpython3/bin/Debug/net6.0/../../../Src/StdLib/Lib/site.py", line 281, in addusersitepackages
  File "/Users/sascha/Dev/ironpython3/bin/Debug/net6.0/../../../Src/StdLib/Lib/site.py", line 257, in getusersitepackages
  File "/Users/sascha/Dev/ironpython3/bin/Debug/net6.0/../../../Src/StdLib/Lib/site.py", line 247, in getuserbase
  File "/Users/sascha/Dev/ironpython3/Src/StdLib/Lib/sysconfig.py", line 595, in get_config_var
  File "/Users/sascha/Dev/ironpython3/Src/StdLib/Lib/sysconfig.py", line 575, in get_config_vars
  File "/Users/sascha/Dev/ironpython3/Src/StdLib/Lib/_osx_support.py", line 393, in customize_config_vars
  File "/Users/sascha/Dev/ironpython3/Src/StdLib/Lib/_osx_support.py", line 135, in _supports_universal_builds
  File "/Users/sascha/Dev/ironpython3/Src/StdLib/Lib/_osx_support.py", line 103, in _get_system_version
LookupError: unknown encoding: cp0IronPython 3.4.0b1 DEBUG (3.4.0.0010)
[.NETCoreApp,Version=v6.0 on .NET 6.0.7 (64-bit)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

When printing out the current encoding, utf_8 is returned:

➜  ironpython3 git:(35cc2274) ✗ bin/Debug/net6.0/ipy -c "import sys; print('The enncoding is='+sys.stdin.encoding)"
Traceback (most recent call last):
  File "<string>", line 1567, in load_module
  File "<string>", line 596, in _load_module_shim
  File "<string>", line 1210, in load
  File "<string>", line 1190, in _load_unlocked
  File "<string>", line 1119, in _exec
  File "<string>", line 1461, in exec_module
  File "<string>", line 539, in _check_name_wrapper
  File "<string>", line 321, in _call_with_frames_removed
  File "/Users/sascha/Dev/ironpython3/bin/Debug/net6.0/../../../Src/StdLib/Lib/site.py", line 564, in <module>
  File "/Users/sascha/Dev/ironpython3/bin/Debug/net6.0/../../../Src/StdLib/Lib/site.py", line 550, in main
  File "/Users/sascha/Dev/ironpython3/bin/Debug/net6.0/../../../Src/StdLib/Lib/site.py", line 281, in addusersitepackages
  File "/Users/sascha/Dev/ironpython3/bin/Debug/net6.0/../../../Src/StdLib/Lib/site.py", line 257, in getusersitepackages
  File "/Users/sascha/Dev/ironpython3/bin/Debug/net6.0/../../../Src/StdLib/Lib/site.py", line 247, in getuserbase
  File "/Users/sascha/Dev/ironpython3/Src/StdLib/Lib/sysconfig.py", line 595, in get_config_var
  File "/Users/sascha/Dev/ironpython3/Src/StdLib/Lib/sysconfig.py", line 575, in get_config_vars
  File "/Users/sascha/Dev/ironpython3/Src/StdLib/Lib/_osx_support.py", line 393, in customize_config_vars
  File "/Users/sascha/Dev/ironpython3/Src/StdLib/Lib/_osx_support.py", line 135, in _supports_universal_builds
  File "/Users/sascha/Dev/ironpython3/Src/StdLib/Lib/_osx_support.py", line 103, in _get_system_version
LookupError: unknown encoding: cp0The enncoding is=utf_8

Looking into the code reveals that f = open('/System/Library/CoreServices/SystemVersion.plist') is raising the error.

➜  ironpython3 git:(35cc2274) ✗ cat /System/Library/CoreServices/SystemVersion.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>ProductBuildVersion</key>
	<string>19H2026</string>
	<key>ProductCopyright</key>
	<string>1983-2022 Apple Inc.</string>
	<key>ProductName</key>
	<string>Mac OS X</string>
	<key>ProductUserVisibleVersion</key>
	<string>10.15.7</string>
	<key>ProductVersion</key>
	<string>10.15.7</string>
	<key>iOSSupportVersion</key>
	<string>13.6</string>
</dict>
</plist>

➜  ironpython3 git:(35cc2274) ✗ file /System/Library/CoreServices/SystemVersion.plist
/System/Library/CoreServices/SystemVersion.plist: XML 1.0 document text, ASCII text

Reading this file with the default encoding yields to the error:

>>> f = open('/System/Library/CoreServices/SystemVersion.plist')
>>> print(f.readlines())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
LookupError: unknown encoding: cp0
>>> f = open('/System/Library/CoreServices/SystemVersion.plist', encoding='utf-8')
>>> print(f.readlines())
['<?xml version="1.0" encoding="UTF-8"?>\n', '<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">\n', '<plist version="1.0">\n', '<dict>\n', '\t<key>ProductBuildVersion</key>\n', '\t<string>19H2026</string>\n', '\t<key>ProductCopyright</key>\n', '\t<string>1983-2022 Apple Inc.</string>\n', '\t<key>ProductName</key>\n', '\t<string>Mac OS X</string>\n', '\t<key>ProductUserVisibleVersion</key>\n', '\t<string>10.15.7</string>\n', '\t<key>ProductVersion</key>\n', '\t<string>10.15.7</string>\n', '\t<key>iOSSupportVersion</key>\n', '\t<string>13.6</string>\n', '</dict>\n', '</plist>\n']

Setting PYTHONIOENCODING did not change the behavior:

➜  ironpython3 git:(35cc2274) ✗ PYTHONIOENCODING=utf_8 bin/Debug/net6.0/ipy
Traceback (most recent call last):
  File "<string>", line 1567, in load_module
  File "<string>", line 596, in _load_module_shim
  File "<string>", line 1210, in load
  File "<string>", line 1190, in _load_unlocked
  File "<string>", line 1119, in _exec
  File "<string>", line 1461, in exec_module
  File "<string>", line 539, in _check_name_wrapper
  File "<string>", line 321, in _call_with_frames_removed
  File "/Users/sascha/Dev/ironpython3/bin/Debug/net6.0/../../../Src/StdLib/Lib/site.py", line 564, in <module>
  File "/Users/sascha/Dev/ironpython3/bin/Debug/net6.0/../../../Src/StdLib/Lib/site.py", line 550, in main
  File "/Users/sascha/Dev/ironpython3/bin/Debug/net6.0/../../../Src/StdLib/Lib/site.py", line 281, in addusersitepackages
  File "/Users/sascha/Dev/ironpython3/bin/Debug/net6.0/../../../Src/StdLib/Lib/site.py", line 257, in getusersitepackages
  File "/Users/sascha/Dev/ironpython3/bin/Debug/net6.0/../../../Src/StdLib/Lib/site.py", line 247, in getuserbase
  File "/Users/sascha/Dev/ironpython3/Src/StdLib/Lib/sysconfig.py", line 595, in get_config_var
  File "/Users/sascha/Dev/ironpython3/Src/StdLib/Lib/sysconfig.py", line 575, in get_config_vars
  File "/Users/sascha/Dev/ironpython3/Src/StdLib/Lib/_osx_support.py", line 393, in customize_config_vars
  File "/Users/sascha/Dev/ironpython3/Src/StdLib/Lib/_osx_support.py", line 135, in _supports_universal_builds
  File "/Users/sascha/Dev/ironpython3/Src/StdLib/Lib/_osx_support.py", line 103, in _get_system_version
LookupError: unknown encoding: cp0IronPython 3.4.0b1 DEBUG (3.4.0.0010)
[.NETCoreApp,Version=v6.0 on .NET 6.0.7 (64-bit)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
➜  ironpython3 git:(35cc2274) ✗ PYTHONIOENCODING=utf-8 bin/Debug/net6.0/ipy
Traceback (most recent call last):
  File "<string>", line 1567, in load_module
  File "<string>", line 596, in _load_module_shim
  File "<string>", line 1210, in load
  File "<string>", line 1190, in _load_unlocked
  File "<string>", line 1119, in _exec
  File "<string>", line 1461, in exec_module
  File "<string>", line 539, in _check_name_wrapper
  File "<string>", line 321, in _call_with_frames_removed
  File "/Users/sascha/Dev/ironpython3/bin/Debug/net6.0/../../../Src/StdLib/Lib/site.py", line 564, in <module>
  File "/Users/sascha/Dev/ironpython3/bin/Debug/net6.0/../../../Src/StdLib/Lib/site.py", line 550, in main
  File "/Users/sascha/Dev/ironpython3/bin/Debug/net6.0/../../../Src/StdLib/Lib/site.py", line 281, in addusersitepackages
  File "/Users/sascha/Dev/ironpython3/bin/Debug/net6.0/../../../Src/StdLib/Lib/site.py", line 257, in getusersitepackages
  File "/Users/sascha/Dev/ironpython3/bin/Debug/net6.0/../../../Src/StdLib/Lib/site.py", line 247, in getuserbase
  File "/Users/sascha/Dev/ironpython3/Src/StdLib/Lib/sysconfig.py", line 595, in get_config_var
  File "/Users/sascha/Dev/ironpython3/Src/StdLib/Lib/sysconfig.py", line 575, in get_config_vars
  File "/Users/sascha/Dev/ironpython3/Src/StdLib/Lib/_osx_support.py", line 393, in customize_config_vars
  File "/Users/sascha/Dev/ironpython3/Src/StdLib/Lib/_osx_support.py", line 135, in _supports_universal_builds
  File "/Users/sascha/Dev/ironpython3/Src/StdLib/Lib/_osx_support.py", line 103, in _get_system_version
LookupError: unknown encoding: cp0IronPython 3.4.0b1 DEBUG (3.4.0.0010)
[.NETCoreApp,Version=v6.0 on .NET 6.0.7 (64-bit)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

Forcefully editing _osx_support.py to open the file with utf-8 works:

f = open('/System/Library/CoreServices/SystemVersion.plist', encoding='utf-8')

Any suggested work-around or fix?

Environment

➜  ~ dotnet --info
.NET SDK (reflecting any global.json):
 Version:   6.0.302
 Commit:    c857713418

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  10.15
 OS Platform: Darwin
 RID:         osx.10.15-x64
 Base Path:   /usr/local/share/dotnet/sdk/6.0.302/

global.json file:
  Not found

Host:
  Version:      6.0.7
  Architecture: x64
  Commit:       0ec02c8c96

.NET SDKs installed:
  6.0.302 [/usr/local/share/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.7 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 6.0.7 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]

x = 111111111111111111111111111111
assert round(x) == x
assert round(x, 2) == x

See https://github.com/IronLanguages/ironpython3/blob/master/WhatsNewInPython31.md

Also round with no ndigits argument always returns an int:

x = 1e16
assert type(round(x)) == type(int(x)) # using type(int(x)) since we haven't unified int/long yet

Implements PEP 383 and Issue #2. Works with all .NET encodings: ASCII, Latin1, UTF-8, UTF-7, UTF-16, UTF-32, etc.

The challenge is that .NET encoder and decoder fallback interfaces are only character oriented. This was also a problem in CPython, but they could extend the interface to allow fallbacks to produce output on byte level (see discussion PEP 383).

Implemented:

• even rounding for floats and ints, floats is partially handled by an update to the DLR submodule.
• all of the various error messages I could find are identical to those in Python 3.
• for types that implement round, round() delegates to this special member.
• when number is an int it returns an int when it can. when it cannot a BigInteger is returned.
• when number is a BigInteger a BigInteger is always returned.

The algorithm I have used for integral rounding is the one described here https://bugs.python.org/msg78141, which I believe is the algorithm that they use in CPython. There is a caveat that when ndigits is negative and cannot be converted to an int, then 0 is returned. There is no concept of signed zero for BigInteger.

I've routed all of the two parameter integral rounding through a single BigInteger method to reduce the total number of code paths. Since there is a need to unify the int and long types and the int method can overflow I think the benefits of this outweigh any immediate performance concerns.

I think I've tested all of the code paths, but I wouldn't say that the numerical testing is bulletproof.

After exiting an except block, no longer treat the latest exception as being "handled." Maintain the exception context when executing via a generator.

Resolves #245.

It's still missing some major parts, but this is a minimal change that seems to get the tests described in the issue passing.

Fixes #50.

changed from int to BigInteger to support ranges beyond 2^64
added applicable test from stdlib
added convenience methods to iptest/assert_util.py to mimic unittest
added slice operator
added == != < <= > >= operators
added __contains__ method
added count and index methods of collections.abc.Sequence
changed start, stop and step to be attributes
added __hash__
added support for objects implementing __index__

Outstanding: pickle testing - it depends on itertools which depends on missing language features I also made a change to Builtin (int is BigInteger) which may need to wait until a real int->long unification is done. When reviewing, go for second commit otherwise rename obfuscates changes.

The result test out of a branch with some other temporary work around:

rejap@WIN-CUE1I6EN9JB ~/github/ironpython3/bin/Debug
$ ./IronPythonTestConsole.exe  IronPythonTest -noh -labels -full -test:IronPythonTest.Cases.IronPythonCases.test_range                 Runtime Environment -
    OS Version: Microsoft Windows NT 6.1.7601 Service Pack 1
  .NET Version: 4.0.30319.18444

***** IronPythonTest.DLL
***** IronPythonCases
***** IronPythonTests
***** test_range
>>>   0.00s testing test_attributes_from_stdlib
>>>   0.25s testing test_comparison_from_stdlib
>>>   0.45s testing test_contains_from_stdlib
>>>   0.51s testing test_count_from_stdlib
>>>   0.58s testing test_empty_from_stdlib
>>>   0.58s testing test_index_from_stdlib
>>>   0.71s testing test_invalid_invocation_from_stdlib
>>>   0.94s testing test_issue11845_from_stdlib
>>>   1.04s testing test_odd_bug_from_stdlib
>>>   1.05s testing test_range
>>>   1.10s testing test_range_collections
>>>   1.12s testing test_range_corner_cases
>>>   1.17s testing test_range_coverage
>>>   1.21s testing test_range_equal
>>>   1.21s testing test_range_from_stdlib
>>>   1.26s testing test_repr_from_stdlib
>>>   1.26s testing test_reverse_iteration_from_stdlib
>>>   1.29s testing test_slice_from_stdlib
>>>   1.33s testing test_strided_limits_from_stdlib
>>>   1.34s testing test_types_from_stdlib
>>>   1.47s testing test_user_index_method_from_stdlib

21 tests passed
Tests run: 1, Passed: 1, Errors: 0, Failures: 0, Inconclusive: 0
  Not run: 0, Invalid: 0, Ignored: 0, Skipped: 0
Elapsed time: 00:00:11.9180000

... Kind of.

I have a couple pieces of the implementation that I think need revision, namely:

1. When I added weakref support to memoryview, for some reason it wasn't releasing in time for the net4.5 target when run through make.ps1 tests. If I ran test_memoryview in any other circumstance the weakref seemed to release appropriately. I left it out for now, but it might be a good idea to investigate, given that we've seen something similar in #544.
2. The change to suboffsets (empty tuple instead of None) should probably be handled on the other side of the IBufferProtocol instead of in memoryview, right?

I'd like to contribute to this project. It seems like documentation is a little sparse, so it's tough to get a sense of what currently works and what doesn't, and what the current goals are.

Here are some examples:

class test(dict):
    def __iter__(self):
        return super().__iter__()
    def keys(self):
        return ["A"]
    def __getitem__(self, key):
        return 1

A = {"A": 1}
B = test({"B": 2})

assert dict(B) == A
assert dict(**B) == A

def fn(**kw):
    return kw

assert fn(**B) == A

d = {}
d.update(B)
assert d == A

d = {}
d.update(**B)
assert d == A

This is causing an issue with order preserving dicts and collections.OrderedDict.

Prerequisites

• [X] Are you running the latest version?
• [X] Are you reporting to the correct repository?
• [X] Did you perform a cursory search?

Description

Type str implementation is missing casefold method. This method was added in Python 3.3

Steps to Reproduce

1. Run ipy.exe
2. Type str.casefold and hit enter
3. See exception

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'type' object has no attribute 'casefold'

Expected behavior: [What you expected to happen]

Runtime should find the method and repor <method 'casefold' of 'str' objects>

Actual behavior: [What actually happened]

Runtime reports AttributeError

Versions

You can get this information from executing ipy -VV.

IronPython 3.4.0 (3.4.0.1000)
[.NETFramework,Version=v4.6.2 on .NET Framework 4.8.4515.0 (64-bit)]

and

IronPython 3.4.0 (3.4.0.1000)
[.NETCoreApp,Version=v6.0 on .NET 6.0.5 (64-bit)]

The following builtin methods have doctests included in their __doc__ string:

• bin
• float.as_integer_ratio
• float.fromhex
• float.hex
• hex
• int
• int.bit_length
• oct

%-formatting of 999999999999998 is busted (outputs f-1) on .NET Framework (because it's rounding up to 1000000000000000 and the formatting logic fails to handle that properly).

assert "%.15g" % 999999999999998 == "999999999999998"

The "new-style" formatting is also failing, but with a ValueError:

assert f"{999999999999998:.15g}" == "999999999999998"

ValueError: Precision not allowed in integer format specifier

Description

print - whether in conjunction with round or not - does not yield the expected output of decimal values as in 2.7

Steps to Reproduce

While I first observed the behavior in some breaking Unit tests while porting my .NET application to IronPython 3.4.0 using the new nupkg, a simple console example should do:

IronPython 3.4.0 (3.4.0.1000)
[.NETFramework,Version=v4.6.2 on .NET Framework 4.8.4515.0 (64-bit)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> print(685.86)
685.86000000000001
>>> f = float(685.86)
>>> print(f)
685.86000000000001
>>> print(round(f,2))
685.86000000000001
>>> print(round(f))
686
>>> print(round(f,1))
685.89999999999998
>>>

Therefore, I'd think of an output rather than a numeric issue.

Expected behavior: While I am aware that not every decimal number can be rendered in full fidelity, I would expect the same output as was the input argument; or in case of round the output being truncated to the specified number of decimal places. This was the case in IronPython 2.7.

Actual behavior: Although the result may be correct from a numeric point of view, the numbers always seem to be output with the same number of decimal places, which is never the one provided as an argument. Especially, rounding to 1 digit should not give me 14 digits.

Versions

• 3.4.0 Release

If this is indeed expected, then I am not sure how to produce a normalized output.