I only have one account and a telnet localhost 4002 gives me a shell as I would expect. Sadly I could not find a way to download and try an older version of the gateway. The current version for linux is 981.3c.

API connection failed: TimeoutError()
---------------------------------------------------------------------------
TimeoutError                              Traceback (most recent call last)
<ipython-input-10-13991e279071> in <module>
      1 con = ib.IB()
----> 2 con.connect('localhost', 4002, clientId=1)

~/.local/lib/python3.8/site-packages/ib_insync/ib.py in connect(self, host, port, clientId, timeout, readonly, account)
    269             account: Main account to receive updates for.
    270         """
--> 271         return self._run(self.connectAsync(
    272             host, port, clientId, timeout, readonly, account))
    273 

~/.local/lib/python3.8/site-packages/ib_insync/ib.py in _run(self, *awaitables)
    308 
    309     def _run(self, *awaitables: Awaitable):
--> 310         return util.run(*awaitables, timeout=self.RequestTimeout)
    311 
    312     def waitOnUpdate(self, timeout: float = 0) -> bool:

~/.local/lib/python3.8/site-packages/ib_insync/util.py in run(timeout, *awaitables)
    320         globalErrorEvent.connect(onError)
    321         try:
--> 322             result = loop.run_until_complete(task)
    323         except asyncio.CancelledError as e:
    324             raise globalErrorEvent.value() or e

~/.local/lib/python3.8/site-packages/nest_asyncio.py in run_until_complete(self, future)
     68                 raise RuntimeError(
     69                     'Event loop stopped before Future completed.')
---> 70             return f.result()
     71 
     72     def _run_once(self):

/usr/lib/python3.8/asyncio/futures.py in result(self)
    176         self.__log_traceback = False
    177         if self._exception is not None:
--> 178             raise self._exception
    179         return self._result
    180 

/usr/lib/python3.8/asyncio/tasks.py in __step(***failed resolving arguments***)
    278                 # We use the `send` method directly, because coroutines
    279                 # don't have `__iter__` and `__next__` methods.
--> 280                 result = coro.send(None)
    281             else:
    282                 result = coro.throw(exc)

~/.local/lib/python3.8/site-packages/ib_insync/ib.py in connectAsync(self, host, port, clientId, timeout, readonly, account)
   1624         try:
   1625             # establish API connection
-> 1626             await self.client.connectAsync(host, port, clientId, timeout)
   1627 
   1628             # autobind manual orders

~/.local/lib/python3.8/site-packages/ib_insync/client.py in connectAsync(self, host, port, clientId, timeout)
    216                 b' ' + self.connectOptions if self.connectOptions else b''))
    217             self.conn.sendMsg(msg)
--> 218             await asyncio.wait_for(self.apiStart, timeout)
    219             self._logger.info('API connection ready')
    220         except Exception as e:

/usr/lib/python3.8/asyncio/tasks.py in wait_for(fut, timeout, loop)
    499             # See https://bugs.python.org/issue32751
    500             await _cancel_and_wait(fut, loop=loop)
--> 501             raise exceptions.TimeoutError()
    502     finally:
    503         timeout_handle.cancel()

I am using IB.client.getReqId() to place my version of bracket orders and set parentId in target and stop loss orders.

But I have stumbled upon an error:

Traceback (most recent call last):
  (...)
  File "_", line 53, in bracket_order
    parent_trade: Trade = ib_client.placeOrder(contract, parent)
  File "/venv/lib/python3.6/site-packages/ib_insync/ib.py", line 528, in placeOrder
    assert trade.orderStatus.status not in OrderStatus.DoneStates
AssertionError

Which means that trade with such ID was found.

Parent order is created like that:

parent = MarketOrder(
    action, quantity,
    orderId=ib_client.client.getReqId(),
    transmit=False,
    conditionsIgnoreRth=False,
    outsideRth=False
)

Entire bracket_order function is copy of ib_insync.ib.IB.bracketOrder source with parent placed as MarketOrder instead of LimitOrder. So all but stopLoss are transmit=False. Orders are placed in order:

• parent,
• target,
• stop.

Is there a problem with IB that it start IB.client on wrong ID or does not update it properly?

Noticed that IB().connect() always fails with asyncio.exceptions.TimeoutError the very first time it is called upon a fresh IBG/TWS start / re-start. The error occurs with and without providing timeout in connect.

The second time on it works without an issue.

Error traceback

Traceback (most recent call last):
  File "c:\Users\User\Documents\Business\Projects\learn\ib\main.py", line 142, in <module>
    all_base(MARKET, ip)
  File "c:\Users\User\Documents\Business\Projects\learn\ib\base.py", line 555, in all_base
    with IB().connect(HOST, PORT, CID, timeout=2) as ib:
  File "C:\Users\User\AppData\Local\Programs\Python\Python38-32\lib\site-packages\ib_insync\ib.py", line 260, in connect
    return self._run(self.connectAsync(
  File "C:\Users\User\AppData\Local\Programs\Python\Python38-32\lib\site-packages\ib_insync\ib.py", line 292, in _run
    return util.run(*awaitables, timeout=self.RequestTimeout)
  File "C:\Users\User\AppData\Local\Programs\Python\Python38-32\lib\site-packages\ib_insync\util.py", line 311, in run
    result = loop.run_until_complete(task)
  File "C:\Users\User\AppData\Local\Programs\Python\Python38-32\lib\asyncio\base_events.py", line 612, in run_until_complete
    return future.result()
  File "C:\Users\User\AppData\Local\Programs\Python\Python38-32\lib\site-packages\ib_insync\ib.py", line 1619, in connectAsync
    await asyncio.wait_for(connect(), timeout or None)
  File "C:\Users\User\AppData\Local\Programs\Python\Python38-32\lib\asyncio\tasks.py", line 490, in wait_for
    raise exceptions.TimeoutError()
asyncio.exceptions.TimeoutError

Probable causes:

Investigating the traceback:

1. It looks like it is caused by _run in ib.py, which somehow misses the *awaitables in def run of util.py that gets into run_until_complete of base_events.py and falls through to line:612 (unreachable by pylance) to return future.result()

2. ib.py seems to take this result forward to tasks.py to raise the exceptions.TimeoutError(). There is a comment in tasks.py pointing to a python bug, that seems to have been abandoned in May 2018. However, on August 27, there seems to have been some changes made in the code base to address fut.cancel.

Queries

• Does this happen to others too?
• Is there something that can be done to address this in ib_insync?

Relates to #274.

Parse the millisecond time stamps received from genericTick 77 and 48 convert it to float seconds and save to a new attribute Ticker.last_fill_time.

This field is absolutely essential for anykind of accurate charting on small time frames.

I'm not married to any particular name for this attribute, some alternatives:

• last_clearing_time
• last_volume_time
• last_trade_time

Note, I also dropped a duplicate code block for tick 77 which wasn't being executed ever due to the elif.

I would like to also add some basic tests for this and an example in the docs if that's ok @erdewit.

I was thinking that the example you provided in #262 using asyncio.run() would be a good start.

For testing I was thinking of starting a suite that can be run by anyone with a data subscription locally. We could do something simple like add some more examples and run them from pytest as well as include them using sphinx's ..literalinclude directive directly from the examples/ directory into the documentation.

What do you think about working towards using a docker container (choose one from #261 ?) in CI to run tests on commits down the road?

TODO:

• [x] change name to rtTime as per @erdewit request and naming consistency
• [ ] doc string documentation at IB.reqMktData for ID=233
• [ ] make it a datetime. I'm personally really opposed to this.

First of all, thank you for sharing this repository. It has saved me days of work and stress that would be put to making sense of the original IB Python API documentation and code.

I'm currently attempting and failing to run examples/qt_ticker_table.py The QtWindow completely freezes and stops responding while running the main loop IB.run()

After about 5 minutes, the following message gets printed (spammed):

QEventDispatcherWin32::wakeUp: Failed to post a message (Not enough quota is available to process this command.)
QEventDispatcherWin32::wakeUp: Failed to post a message (Not enough quota is available to process this command.)
QEventDispatcherWin32::wakeUp: Failed to post a message (Not enough quota is available to process this command.)
QEventDispatcherWin32::wakeUp: Failed to post a message (Not enough quota is available to process this command.)
....
QEventDispatcherWin32::wakeUp: Failed to post a message (Not enough quota is available to process this command.)

I'm running Windows 10. I tried using both PyQt5 and PySide2 (including changing the respective imports and util.useQt('PySide2) to no avail.

Any advice on how to make this work?

It seems this is an issue in TWS itself, but I provide a description here anyhow as it could also be an interaction effect between TWS and ib_insync.

Since updating TWS to Build 10.16.1f, Jun 2, 2022 4:45:09 PM, my standard loop code breaks. The loop is typically a forever loop with a waitOnUpdate(0.1) to more or less match the rate of ticker update. The code still works running with the gateway.

After digging, it seems the problem is in reqCurrentTime(), everything else seems to work fine, and specifically when calling reqCurrentTime() faster than once per second. I have no good idea why this would happen. Here is the smallest reproducible code that triggers the behavior

delay = 1.1 for i in range(10): print(f"Loop {i}, delay {delay:.1f}") print(f"{ib.reqCurrentTime()}") ib.waitOnUpdate(delay) delay = max(0.1, delay - 0.1)

Running this code result in this output, with the code never returning from reqCurrentTime()

Loop 0, delay 1.1 2022-06-06 05:09:25+00:00 Loop 1, delay 1.0 2022-06-06 05:09:26+00:00 Loop 2, delay 0.9 2022-06-06 05:09:27+00:00 Loop 3, delay 0.8

When that happens in a normal loop, log message linked to event handler still happens, for example a function called at bar updates still work.

Hi, there. I am very interested in your project. I am going to integrate tradingview webhook and ib_insync. Can you help me? I will wait for your favorable reply. Thanks.

@erdewit first I want to thank you immensely for this project :smile_cat:

I've dug through the messaging machinery in client.py and I have to say, thank you so much for allowing the world to avoid having to use the monstrosity that is ibapi.client.EClient :rocket:

The point of this issue is to suggest a change and possibly start a discussion surrounding an alternative method for giving users a synchronous api around ib_insync.

I realize the library is supposed to equip users unfamiliar with asynchronous systems but at the moment the approach used is disadvantageous to users who are familiar with async programming and in particular asyncio and the beauty that normally comes with the ability to leverage that ecosystem for data streaming purposes.

My proposal for the sync API

• run the asyncio event loop in a separate background thread
• have synchronous IB API (methods) schedule async calls and receive results using asyncio.run_coroutine_threadsafe()
• eventually consider use of asyncio.run() as the thread entry point (presuming of course that there is an eventual move to 3.7+)

Why to change this

• the main reason is to avoid user code being able to ever block the asyncio loop, effectively masquerading as though it is synchronous code while constantly starting and stopping the event loop using util.run()/util.sleep(), i.e.
  • there will be no need for IB.sleep() calls by "synchronous" user code as is described by:

  The one rule when working with the IB class is therefore that user code may not block for too long.

  • user code will never be able to slow down or unnecessarily block the ib_insync protocol engine resulting in the ability to use deterministic timing logic in async code
  • if there is a reason to rate limit IB requests due to api restrictions on the IB server side, ib_insync asynchronous code will never have to worry about erroneous user code blocking unecessarily
  • backpressure logic for streaming can be delegated to appropriate asyncio data structures instead of haphazardly managed in user functions invoked as callbacks
• asyncio knowledged programmers will be much happier and able to leverage forwarding streams of real-time data from IB using async functions (which of course asyncio now allows scheduling with .create_task()) and the streaming ecosystem instead of relying on synchronous functions invoked as callbacks.
• async programmers will be able to run the engine in an async proper way (where the user is capable and in charge of scheduling their own tasks on the loop) without any background thread.
• there will be no performance penalty; I know this personally from having written a similar protocol-oriented-client type system which was deployed at scale.
• it should allow for true concurrent api requests to IB's endpoints; eg. getting multiple ContractDetails in "parallel" (in the i/o sense of course) even from the synchronous API.
• it will make integration with other foreign event loops (trio, curio) in Python's async space much easier.
  • woops, this might be my main reason for this request :detective:

My apologies if this was a bit long but I'd like to at least open discussion and get your thoughts.

Thanks :)

Hey Ewald, @erdewit , there is a TRIN-NYSE, TICK-NYSE problem with reqRealTimeBars(contract, 30, 'MIDPOINT', False). Other indices work just fine.

The error returned by IB:

Error 321, reqId 69: Error validating request.-'bT' : cause - Please enter exchange, contract: Index(symbol='TRIN-NYSE') Error 10225, reqId 29: Bust event occurred, current subscription is deactivated. Please resubscribe real-time bars immediately., contract: Index(conId=26718743, symbol='TRIN-NYSE', exchange='NYSE', currency='USD', localSymbol='TRIN-NYSE')

p.s.: thx for ur work!

• Ticker.lastSize is float
• TickData.size is int

Is there a reason to do with different asset types?

I'm also noticing some duplicate code where this seems to be set in Wrapper.tickString():

• block for handling tick types 48 and 77 where Ticker.lastSize is set as an int
• block for handling tick types 77 where Ticker.lastSize is also set as an int

Actually these two blocks look almost identical minus this little nested branch section. I'm pretty sure these could be factored.

I'm only looking at this because I realized we're always discarding the RT Volume, clearing time from the exchange.

This pairs with questions I had in #272. Normally i would expect tick data to have the exchange clearing time for any trades/volume.

As an example (after I've patched the code - don't worry I'll make a PR) this is what I now see from my streaming tick data after adding a last_fill_time field from the exchange clearing time we are discarding:

trio got: {'time': 1593468211.4448721, 'tickType': 3, 'price': 3046.5, 'size': 4, 'last_fill_time': None}
trio got: {'time': 1593468211.6727612, 'tickType': 77, 'price': 3046.25, 'size': 7, 'last_fill_time': 1593468211.081}
trio got: {'time': 1593468211.6734376, 'tickType': 3, 'price': 3046.5, 'size': 7, 'last_fill_time': None}
trio got: {'time': 1593468211.6734376, 'tickType': 3, 'price': 3046.5, 'size': 7, 'last_fill_time': None}
trio got: {'time': 1593468211.6734376, 'tickType': 3, 'price': 3046.5, 'size': 7, 'last_fill_time': None}
trio got: {'time': 1593468211.872757, 'tickType': 77, 'price': 3046.25, 'size': 1, 'last_fill_time': 1593468211.281}
trio got: {'time': 1593468211.872757, 'tickType': 77, 'price': 3046.25, 'size': 1, 'last_fill_time': 1593468211.281}
trio got: {'time': 1593468212.3658435, 'tickType': 48, 'price': 3046.25, 'size': 1, 'last_fill_time': 1593468211.774}
trio got: {'time': 1593468212.3662589, 'tickType': 77, 'price': 3046.25, 'size': 1, 'last_fill_time': 1593468211.774}
trio got: {'time': 1593468212.3925517, 'tickType': 77, 'price': 3046.25, 'size': 8, 'last_fill_time': 1593468211.801}
trio got: {'time': 1593468212.3925517, 'tickType': 77, 'price': 3046.25, 'size': 8, 'last_fill_time': 1593468211.801}
trio got: {'time': 1593468212.6746254, 'tickType': 8, 'price': -1.0, 'size': 1699, 'last_fill_time': None}
trio got: {'time': 1593468212.6753259, 'tickType': 0, 'price': 3046.25, 'size': 5, 'last_fill_time': None}
trio got: {'time': 1593468212.6753259, 'tickType': 3, 'price': 3046.5, 'size': 8, 'last_fill_time': None}
trio got: {'time': 1593468212.6753259, 'tickType': 0, 'price': 3046.25, 'size': 5, 'last_fill_time': None}
trio got: {'time': 1593468212.6753259, 'tickType': 3, 'price': 3046.5, 'size': 8, 'last_fill_time': None}

Hey @drusakov778, look you can actually see the exchange -> ib_insync latency now :wink:.

Hi, I'm trying to integrate ib-insync with django using django-channels to get real time data from interactive brokers using a socket and using reqTickByTickData. But I'm struggling with the initial connection.

I wrote this line of code in services.py:

import json, nest_asyncio, asyncio
from ib_insync import *
ib = IB()
def connect_to_ib(): 
    print("Starting IB")
    ib_connection = ib.connect('127.0.0.1', 4002, clientId=1,timeout=4, readonly=False, account='')
    print("IB is Connected?", ib.isConnected())
    return ib_connection

but I'm getting this error:

RuntimeError: There is no current event loop in thread 'django-main-thread'

It worked in the past briefly and get an ib Object but I don't know why It fails now.

I read #33 and #167 in order to solve this error but the problem persist.

def create_evenloop():
    print("Starting Event Loop")
    loop = asyncio.new_event_loop()
    asyncio.set_event_loop(loop)
    nest_asyncio.apply()

Those functions were called in a consumer in django channels, (normally django is blocking but AsyncWebsocketConsumer from channels make non-blocking code inside PlotterConsumer class )

class PlotterConsumer(AsyncWebsocketConsumer):
    create_evenloop()
    ib = connect_to_ib()
    ...

from ib_insync import * #util.startLoop()

ib = IB() ib.connect('127.0.0.1', 7497, clientId=123)

def getMktData(symbol): contract = Stock(symbol, 'SMART', 'USD') bars = ib.reqHistoricalData( contract, endDateTime='', durationStr='3 M', barSizeSetting='10 mins', whatToShow='TRADES', useRTH=True, formatDate=1) df = util.df(bars) return df

rst=getMktData('KWEB')

output

The contract description specified for KWEB is ambiguous., contract: Stock(symbol='KWEB', exchange='SMART', currency='USD')

I've been facing an issue where a connection between a code that runs in WSL2 (ubuntu 22.04) and a TWS running on Windows 10 is disconnected due to timeout.

Connecting to the same TWS using the same code but directly from Windows (without WSL) works flawlessly.

In TWS settings, I have enabled connections from 172.18.240.1.

Log is as follows

2022-11-21 19:24:14,360 ib_insync.client INFO Connecting to 172.18.240.1:7496 with clientId 999...
2022-11-21 19:24:14,362 ib_insync.client INFO Connected
2022-11-21 19:24:18,363 ib_insync.client INFO Disconnecting
2022-11-21 19:24:18,364 ib_insync.client ERROR API connection failed: TimeoutError()

ib-insync 0.9.71 Python 3.10.4 Demo account Trader Workstation Build 10.18.1e, Sep 20, 2022 4:11:21 PM Windows10 1809

Found Bug: only first order appears at TWS if ib.sleep(0) added - all orders appears print shows correct OrdedID assigned to each trade

code example:

from ib_insync import *

ib = IB()
ib.connect('127.0.0.1',7497)
contract = Stock (symbol='SPY', exchange='SMART',  currency='USD')

for n in range(3):
    order = MarketOrder('BUY',1,account = ib.wrapper.accounts[0] )
    trade = ib.placeOrder(contract,order)
    print('---',trade)
    # ib.sleep(0)  #uncomment this

ib_insync's filled quantity algorithm performs a sum of the quantity of all the trade's fills: https://github.com/erdewit/ib_insync/blob/20180813c2c768c408f1febe7e1373423dd9d0f0/ib_insync/wrapper.py#L436

These fills are communicated as "executions" by IB through the execDetails callback. However IB API's documentation indicates that an execution's ExecId field might indicate in some cases that it is not a new execution but a correction to a previous execution (https://interactivebrokers.github.io/tws-api/classIBApi_1_1Execution.html): " string ExecId The execution's identifier. Each partial fill has a separate ExecId. A correction is indicated by an ExecId which differs from a previous ExecId in only the digits after the final period, e.g. an ExecId ending in ".02" would be a correction of a previous execution with an ExecId ending in ".01". "

The documentation does not indicate if the correction is a replacement (ie it cancels and replaces the previous execution) or a delta (ie its quantities shall be added to the previous execution's quantities to correct them). ib_insync's code seems to assume the later. Is it a deliberate choice based on observing these cases, or is it a potential bug? I have not observed this case directly in my new logs (my old logs are not usable for such purpose) so maybe this potential bug is pedantic.

The code managing the cancellation of inactive orders (https://github.com/erdewit/ib_insync/blob/006fcfc19bb7578e267096e61ae74fae6aff90e1/ib_insync/ib.py#L681) declares them immediately as Cancelled, as opposed to PendingCancel. There is one common case where this seems premature: unavailable shares to short. In these case, IB declares the order as inactive, but it is still looking for these shares and could at any moment execute the order should new borrowable shares become available (I practice short-selling quite a lot and this is often unpredictable). To handle these cases properly inactive orders shall be removed from this condition, i.e. declared as PendingCancel until the server confirms the effective cancellation.

Note: I have only a cursory understanding of how the library and the IB API interact so I may be completely wrong. For example I don't know if the server does send a cancellation confirmation for inactive orders.