I have used a your project to run genetic programming on google colab, when i run karoo_gp file on google colab i get this error==> ValueError: could not convert string to float: ' "shell_port": 37649'

How can i solve this problem?

Good Day,

In the update of Karoo in 2017, it mentions the inclusion of >,<,<=,>=. However, the latest karoo GP does not seem to accept these functions.

Please excuse me if this question is a repeat, but I am finding it hard to run my project without such functions, will they be included?

I get the following error when trying to run the package, it seems that the version of tensorflow used in the package is an older version:

File "modules\karoo_gp_base_class.py", line 59, in 'log': tf.log, # e.g., log(a) AttributeError: module 'tensorflow' has no attribute 'log'

Hello I got this package working with tensorflow and have started testing out the various features. So far I really love the library, but I found an error trying to test a particular tree from the desktop ui. Not sure what the issue is, but raising it here.

Here's the details.

I installed for linux, on centos. I am using tensorflow-gpu. Version 1.5.0 I installed cudnn-9.1-linux-x64-v7.tgz

check my version of tensorflow: [karoo_gp]$ python -c 'import tensorflow as tf; print(tf.version)' 1.5.0

[karoo_gp]$ python karoo_gp_main.py files/data_CLASSIFY.csv

I select classification, and run with defaults until model completes after the model is done, I select "t" and try to test a model the following error is raised:

Karoo GP has an ellapsed time of 42.032015

"It is not the strongest of the species that survive, nor the most intelligent, but the one most responsive to change." --Charles Darwin

Congrats! Your multi-generational Karoo GP run is complete.

Type ? to review your options or q to quit.

 (pause) ?

Select one of the following options:
 i 	 Interactive display mode
 m 	 Minimal display mode
 g 	 Generation display mode
 s 	 Silent display mode
 db 	 De-Bug display mode

 ts 	 adjust the tournament size
 min 	 adjust the minimum number of nodes
 bal 	 adjust the balance of genetic operators

 l 	 list Trees with leading fitness scores
 t 	 evaluate a single Tree against the test data

 p 	 print a single Tree to screen
 id 	 display the current generation ID
 pop 	 list all Trees in current population
 dir 	 display current working directory

 cont 	 continue evolution, starting with the current population
 load 	 load population_s (seed) to replace population_a (current)
 w 	 write the evolving population_b to disk
 q 	 quit Karoo GP without saving population_b

 Remember to archive your final population before your next run!

 (pause) t

 Select a Tree in population_b to test: 8

Traceback (most recent call last): File "karoo_gp_main.py", line 255, in gp.fx_karoo_eol() File "/home/andrew/gep/karoo_gp/karoo_gp_base_class.py", line 884, in fx_karoo_eol self.fx_karoo_pause(1) File "/home/andrew/gep/karoo_gp/karoo_gp_base_class.py", line 577, in fx_karoo_pause if eol == 1: self.fx_karoo_pause(1) # return to pause menu as the GP run is complete File "/home/andrew/gep/karoo_gp/karoo_gp_base_class.py", line 723, in fx_karoo_pause result = self.fx_fitness_eval(expr, self.data_test, get_labels=True) File "/home/andrew/gep/karoo_gp/karoo_gp_base_class.py", line 1463, in fx_fitness_eval if get_labels: labels = tf.map_fn(self.fx_fitness_labels_map, result, dtype=[tf.int32, tf.string], swap_memory=True) File "/usr/lib/python2.7/site-packages/tensorflow/python/ops/functional_ops.py", line 409, in map_fn swap_memory=swap_memory) File "/usr/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2934, in while_loop result = loop_context.BuildLoop(cond, body, loop_vars, shape_invariants) File "/usr/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2720, in BuildLoop pred, body, original_loop_vars, loop_vars, shape_invariants) File "/usr/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2662, in _BuildLoop body_result = body(*packed_vars_for_body) File "/usr/lib/python2.7/site-packages/tensorflow/python/ops/functional_ops.py", line 400, in compute nest.assert_same_structure(dtype or elems, packed_fn_values) File "/usr/lib/python2.7/site-packages/tensorflow/python/util/nest.py", line 197, in assert_same_structure _recursive_assert_same_structure(nest1, nest2, check_types) File "/usr/lib/python2.7/site-packages/tensorflow/python/util/nest.py", line 156, in _recursive_assert_same_structure % (type_nest1, type_nest2)) TypeError: The two structures don't have the same sequence type. First structure has type <type 'list'>, while second structure has type <type 'tuple'>.

Some of the operators we support will produce unusable values (nan or inf) in the course of normal use:

|Operator |X == 0 | X > 1e3| X < 0 | X > 1 | |----------|--------|--------|--------|--------| |/ | nan* |** | | inf
|sqrt | nan* | | nan |log | -inf | | nan |log1p | | | nan |arcsin | | | | nan |arccos | | | | nan

*We currently use helper functions for division and square root which ignore 0s.

What to do?

Here are 3 ideas:

1. Deal with them case-by-case.

   • / and sqrt seem ok for now.
   • log1p is a built-in function that extends log by ignoring 0s. We could add a helper which does sign(x) * log1p(abs(x)).
   • arccos and arcsin are maybe rare enough, we could add a check in karoo.fit() when using them that -1 < X < 1, else raise a ValueError.
   • That leaves **. X > 1e3 happens frequently with small numbers too when combined with other operators, e.g. 2 ** (1 / .001). Replacing with 0 is the simplest option, but it's a big nonlinearity (as X increases, outputs get exponentially larger and then drop to 0).

2. Accept a kwarg with a replacement value (e.g. 0) in the case that a nan and/or inf is produced. Basically like we do in the *'s above, for everything.

3. If and when a tree produces a nan or inf, just remove it from the gene pool and don't bother scoring it. This is basically the method used by swim, i.e. eliminate trees with less than the minimum number of nodes.

I lean toward 3.

Thanks for your hard work.

1. AS TF, Scikit, packages are evolving a lot, it would be useful to document: For each release of KAROO_GP: Dependency list (TF 1.0.1, .....) of Python packages. Regression test in a folder test_regression: input_data test_01.py

We can check install easily as well as the results.

2. We may want to integrate into other python data processing/code. is there a way to import karoo as a package ? import karoo as ka

3. For some problems, we may want to limit the operation list (ex: cyclical data, we want to remove sin and keep only keep cos(a*t+b) ) Is there a way to add a config file than karoo can read (so we don't have to change the core source).

4. What about custom primitive ? It may help for complex modelling and reducing tree depth. Operator Primitive: exp( -a *x**2)

Ex: Multi-period data (5 <> period) and we split into medium period to simplify the problem.

Hello, I am working with Python 3.5 I have methodically made changes to karoo_gp_base_class.py, karoo_gp_main.py and karoo_gp_server.py to be compatible with Python 3. I have now reached an error that I am unsure how to fix: (After accepting all default options for a run of karoo_gp_main I get)

Traceback (most recent call last): File "karoo_gp_main.py", line 210, in gp.fx_fitness_gym(gp.population_a) # generate expression, evaluate fitness, compare fitness File "C:\Users\shiri\Documents\karoo_gp-master\karoo_gp_base_class.py", line 1351, in fx_fitness_gym print ('\n\033[36m ', len(self.fittest_dict.keys()), 'trees\033[1m', np.sort(self.fittest_dict.keys()), '\033[0,0m\033[36moffer the highest fitness scores.\033[0,0m') File "C:\Anaconda3\envs\py35\lib\site-packages\numpy\core\fromnumeric.py", line 822, in sort a.sort(axis=axis, kind=kind, order=order) numpy.core._internal.AxisError: axis -1 is out of bounds for array of dimension 0

Please advise how to fix this issue

The following command sometimes fails with ERROR! In fx_evolve_grow_mutate: branch_depth -1 < 0:

python3 karoo-gp.py -ker r -typ g -bas 5 -pop 20 -fil karoo_gp/files/data_REGRESS.csv

To reproduce, you can run it in a loop until it fails using:

while true; do python3 karoo-gp.py -ker r -typ g -bas 5 -pop 20 -fil karoo_gp/files/data_REGRESS.csv; done

After the error it stops in the pause menu, which shouldn't happen for this type of runs.

If I run in interactive mode (I've also seen it at the end of a "minimal" mode run), then I can use the pause menu at most once per generation:

  2 trees [ 1 38] offer the highest fitness scores.

         (pause) t
...
 Confusion matrix:
[[3072  984]
 [2997  947]]

         (pause) t
         Select from the options given. Try again ...

         (pause) ?
         Select from the options given. Try again ...

         (pause) p
         Select from the options given. Try again ...

         (pause) 30
         Select from the options given. Try again ...

I'm guessing it's some kind of parsing bug where the state of some buffer is left inconsistent after the first "t" operation. All I can do now is hit enter and let the next generation run, after which I'll be able to use "t" on (at most) one of the winners from that generation.

$ python karoo_gp_main.py
Traceback (most recent call last):
  File "karoo_gp_main.py", line 35, in <module>
    import karoo_gp_base_class; gp = karoo_gp_base_class.Base_GP()
  File "/Users/bill/programming/karoo_gp/karoo_gp_base_class.py", line 116, in __init__
    self.core_count = pp.get_number_of_cores() # pprocess
  File "/usr/local/lib/python2.7/site-packages/pprocess.py", line 939, in _get_number_of_cores
    f = open("/proc/cpuinfo")
IOError: [Errno 2] No such file or directory: '/proc/cpuinfo'

Probably this should have exception handling, and return a default value if an error occurs.

the fact this user guide is exclusively hosted on github makes it impossible to open the file in the mobile browser and thus it is impossible to search without downloading it onto your device and opening it in an app.

Would it be possible to host this pdf somewhere and add a link to the pdf in the readme?

I get the following error: (note that I use anaconda navigator to create an environment with the required versions of the dependent modules) and the following command line and functions:

operator, arity
+,2
-,2
/,2
*,2
<,2
>=,2
or,2
and,2

(macos-TF2) aymansalsaket@AYMANs-MacBook-Pro latest % python -V
Python 3.8.8
(macos-TF2) aymansalsaket@AYMANs-MacBook-Pro latest % sudo nice -1 python karoo-gp.pypython -V 

Select (c)lassification, (r)egression, (m)atching, or (p)lay (default m): c
Select (f)ull, (g)row, or (r)amped 50/50 method (default r): r
Enter depth of the initial population of Trees (default 3): **7**
Enter maximum Tree depth (default 7): **7**
Enter minimum number of nodes for any given Tree (default 3; max 255): **5**
Enter number of Trees in each population (default 100): 75
Enter max number of generations (default 10): 100

In the settings above only 5,5,3 work till the end. Here, I used 7,7,5 and it crashed as shown in the trace:

ERROR TRACE:

Evaluate all Trees in Generation 4
ValueError: Error from parse_expr with transformed code:
   "(((Symbol ('A1x' ))>=(Symbol ('A21x' )))<((Symbol ('A11x' ))>=(Symbol ('A2x' ))))"

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "karoo-gp.py", line 593, in <module>
    gp.fit(X, y)
  File "/Users/aymansalsaket/Desktop/latest/karoo_gp/base_class.py", line 804, in fit
    super().fit(X, y, *args, **kwargs)
  File "/Users/aymansalsaket/Desktop/latest/karoo_gp/base_class.py", line 460, in fit
    self.population.evaluate(X_train, y_train, self.X_train_hash)
  File "/Users/aymansalsaket/Desktop/latest/karoo_gp/population.py", line 93, in evaluate
    predictions = self.model.batch_predict(X, self.trees, X_hash)
  File "/Users/aymansalsaket/Desktop/latest/karoo_gp/base_class.py", line 494, in batch_predict
    y = self.engine.predict(trees, X, X_hash)
  File "/Users/aymansalsaket/Desktop/latest/karoo_gp/engine.py", line 155, in predict
    expr = tree.expression
  File "/Users/aymansalsaket/Desktop/latest/karoo_gp/tree.py", line 76, in expression
    return self.root.parse(simplified=True)
  File "/Users/aymansalsaket/Desktop/latest/karoo_gp/node.py", line 203, in parse
    return (f'({self.children[0].parse(simplified)}'
  File "/Users/aymansalsaket/Desktop/latest/karoo_gp/node.py", line 203, in parse
    return (f'({self.children[0].parse(simplified)}'
  File "/Users/aymansalsaket/Desktop/latest/karoo_gp/node.py", line 200, in parse
    return (f'({self.children[0].parse(simplified)}{ws}{self.label}'
  File "/Users/aymansalsaket/Desktop/latest/karoo_gp/node.py", line 200, in parse
    return (f'({self.children[0].parse(simplified)}{ws}{self.label}'
  File "/Users/aymansalsaket/Desktop/latest/karoo_gp/node.py", line 200, in parse
    return (f'({self.children[0].parse(simplified)}{ws}{self.label}'
  File "/Users/aymansalsaket/Desktop/latest/karoo_gp/node.py", line 172, in parse
    result = str(sympify(raw_expr))
  File "/opt/anaconda3/envs/macos-TF2/lib/python3.8/site-packages/sympy/core/sympify.py", line 495, in sympify
    expr = parse_expr(a, local_dict=locals, transformations=transformations, evaluate=evaluate)
  File "/opt/anaconda3/envs/macos-TF2/lib/python3.8/site-packages/sympy/parsing/sympy_parser.py", line 1105, in parse_expr
    raise e from ValueError(f"Error from parse_expr with transformed code: {code!r}")
  File "/opt/anaconda3/envs/macos-TF2/lib/python3.8/site-packages/sympy/parsing/sympy_parser.py", line 1096, in parse_expr
    rv = eval_expr(code, local_dict, global_dict)
  File "/opt/anaconda3/envs/macos-TF2/lib/python3.8/site-packages/sympy/parsing/sympy_parser.py", line 915, in eval_expr
    expr = eval(
  File "<string>", line 1, in <module>
TypeError: '<' not supported between instances of 'GreaterThan' and 'GreaterThan'

In GP, the fittest tree in a generation is, in one scenario, carried forward to the next generation.

With this in mind, the fittest tree in any generation cannot be lower than the inherited tree.

In my tests, I noticed that the fittest tree value drops, which does not conform to the "survival of the fittest" as described by Koza.

I wonder what you think?

Hello,

After doing some testing, I am nearly certain that Sympy & Numpy don't play well together, especially when tree depth and min nodes are higher than 5.

There are many references on stack overflow regarding this issue.

We're planning a major release, Karoo 3.0, that will be pip-installable and include all the recent features - the new API, sklearn-compatibility, logical operators - and some additional items. The purpose of this issue is to determine which items to include, and track our progress toward the release.

Below is my first take:

Karoo 3.0 (include)

• #92
• #83
• #73 Inline-execution and tensorflow 2, per my latest comments
• Documentation with sphinx for readthedocs.io, including the content from the Karoo_GP_User_Guide.pdf updated to be current

Karoo 3.N (later)

• Add click for color in Interactive-mode menu.
• Remove ast from the Tree.load process.
• Basic parsimony (T/F, choose shorter if equal fitness, add to BaseGP.compare_fitness
• Finalize sklearn: pass sklearn.utils.estimator_checks.check_estimator(BaseGP).
• Add support for log and trigonometric funcs.
• Add simplifying support for operators not supported by Sympy.
• Tree.display support highlighting specific nodes
• Replace the current calls to log/pause/error with a 'hooks' system.
• Don't store X and y in the model directly. (needs hooks)
• Make save/load population api more generic, move the population_a.csv references to a dict/enum in karoo-gp.py. (needs hooks)
• Limit 'load' functionality in Interactive mode to certain stages
• Make it possible to save/reload and continue training from same point.
• Create more GP classes: BinaryClassifierGP, EnsembleClassifierGP, BooleanGP, etc.

Hello,

I got a consistent error with one of my setups. The data is error free and works with other operator setups but not this one:

+,2 -,2

=,2 /,2 *,2 or,2

`Evaluate all Trees in Generation 4 ValueError: Error from parse_expr with transformed code: "(((Symbol ('A17x' ))>=(Symbol ('A6x' )))>=((Symbol ('A3x' ))>=(Symbol ('A22x' ))))"

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/Users/aymansalsaket/Desktop/new/karoo-gp.py", line 593, in gp.fit(X, y) File "/Users/aymansalsaket/Desktop/new/karoo_gp/base_class.py", line 802, in fit super().fit(X, y, *args, **kwargs) File "/Users/aymansalsaket/Desktop/new/karoo_gp/base_class.py", line 458, in fit self.population.evaluate(X_train, y_train, self.X_train_hash) File "/Users/aymansalsaket/Desktop/new/karoo_gp/population.py", line 93, in evaluate predictions = self.model.batch_predict(X, self.trees, X_hash) File "/Users/aymansalsaket/Desktop/new/karoo_gp/base_class.py", line 492, in batch_predict y = self.engine.predict(trees, X, X_hash) File "/Users/aymansalsaket/Desktop/new/karoo_gp/engine.py", line 155, in predict expr = tree.expression File "/Users/aymansalsaket/Desktop/new/karoo_gp/tree.py", line 78, in expression return self.root.parse(simplified=True) File "/Users/aymansalsaket/Desktop/new/karoo_gp/node.py", line 317, in parse return (f'({self.children[0].parse(simplified)}{ws}{self.label}' File "/Users/aymansalsaket/Desktop/new/karoo_gp/node.py", line 318, in parse f'{ws}{self.children[1].parse(simplified)})') File "/Users/aymansalsaket/Desktop/new/karoo_gp/node.py", line 318, in parse f'{ws}{self.children[1].parse(simplified)})') File "/Users/aymansalsaket/Desktop/new/karoo_gp/node.py", line 318, in parse f'{ws}{self.children[1].parse(simplified)})') File "/Users/aymansalsaket/Desktop/new/karoo_gp/node.py", line 322, in parse f'{ws}else{ws}{self.children[2].parse(simplified)})') File "/Users/aymansalsaket/Desktop/new/karoo_gp/node.py", line 321, in parse f'{ws}if{ws}{self.children[1].parse(simplified)}' File "/Users/aymansalsaket/Desktop/new/karoo_gp/node.py", line 289, in parse result = str(sympify(raw_expr)) File "/opt/anaconda3/envs/Master-TF/lib/python3.9/site-packages/sympy/core/sympify.py", line 495, in sympify expr = parse_expr(a, local_dict=locals, transformations=transformations, evaluate=evaluate) File "/opt/anaconda3/envs/Master-TF/lib/python3.9/site-packages/sympy/parsing/sympy_parser.py", line 1105, in parse_expr raise e from ValueError(f"Error from parse_expr with transformed code: {code!r}") File "/opt/anaconda3/envs/Master-TF/lib/python3.9/site-packages/sympy/parsing/sympy_parser.py", line 1096, in parse_expr rv = eval_expr(code, local_dict, global_dict) File "/opt/anaconda3/envs/Master-TF/lib/python3.9/site-packages/sympy/parsing/sympy_parser.py", line 915, in eval_expr expr = eval( File "", line 1, in TypeError: '>=' not supported between instances of 'GreaterThan' and 'GreaterThan' `