adds flavor of interactive filtering to the traditional pipe concept of UNIX shell

Masafumi Oyamada

Last update: Jan 7, 2023

Related tags

CLI Tools percol

Overview

percol

                                __
    ____  ___  ______________  / /
   / __ \/ _ \/ ___/ ___/ __ \/ /
  / /_/ /  __/ /  / /__/ /_/ / /
 / .___/\___/_/   \___/\____/_/
/_/

percol adds flavor of interactive selection to the traditional pipe concept on UNIX.

What's this
- Features
- Related projects
Installation
- PyPI
- Manual
Usage
Example
Configuration
- Customizing prompt
  - Dynamic prompt
  - Custom format specifiers
- Customizing styles
Matching Method
Tips
- Selecting multiple candidates
- Z Shell support

What's this

percol is an interactive grep tool in your terminal. percol

receives input lines from stdin or a file,
lists up the input lines,
waits for your input that filter/select the line(s),
and finally outputs the selected line(s) to stdout.

Since percol just filters the input and output the result to stdout, it can be used in command-chains with | in your shell (UNIX philosophy!).

Features

Efficient: With lazy loads of input lines and query caching, percol handles huge inputs efficiently.
Customizable: Through configuration file (rc.py), percol's behavior including prompts, keymaps, and color schemes can be heavily customizable.
- See configuration for details.
Migemo support: By supporting C/Migemo, percol filters Japanese inputs blazingly fast.
- See matching method for details.

Related projects

canything by @keiji0
- A seminal work in interactive grep tools.
zaw by @nakamuray
- A zsh-friendly interactive grep tool.
peco by @lestrrat
- An interactive grep tool written in Go language.
fzf by @junegunn
- An interactive grep tool written in Go language.

Installation

percol currently supports only Python 2.x.

PyPI

$ sudo pip install percol

Manual

First, clone percol repository and go into the directory.

$ git clone git://github.com/mooz/percol.git
$ cd percol

Then, run a command below.

$ sudo python setup.py install

If you don't have a root permission (or don't wanna install percol with sudo), try next one.

$ python setup.py install --prefix=~/.local
$ export PATH=~/.local/bin:$PATH

Usage

Specifying a filename.

$ percol /var/log/syslog

Specifying a redirection.

$ ps aux | percol

Example

Interactive pgrep / pkill

Here is an interactive version of pgrep,

$ ps aux | percol | awk '{ print $2 }'

and here is an interactive version of pkill.

$ ps aux | percol | awk '{ print $2 }' | xargs kill

For zsh users, command versions are here (ppkill accepts options like -9).

function ppgrep() {
    if [[ $1 == "" ]]; then
        PERCOL=percol
    else
        PERCOL="percol --query $1"
    fi
    ps aux | eval $PERCOL | awk '{ print $2 }'
}

function ppkill() {
    if [[ $1 =~ "^-" ]]; then
        QUERY=""            # options only
    else
        QUERY=$1            # with a query
        [[ $# > 0 ]] && shift
    fi
    ppgrep $QUERY | xargs kill $*
}

zsh history search

In your .zshrc, put the lines below.

function exists { which $1 &> /dev/null }

if exists percol; then
    function percol_select_history() {
        local tac
        exists gtac && tac="gtac" || { exists tac && tac="tac" || { tac="tail -r" } }
        BUFFER=$(fc -l -n 1 | eval $tac | percol --query "$LBUFFER")
        CURSOR=$#BUFFER         # move cursor
        zle -R -c               # refresh
    }

    zle -N percol_select_history
    bindkey '^R' percol_select_history
fi

Then, you can display and search your zsh histories incrementally by pressing Ctrl + r key.

tmux

Here are some examples of tmux and percol integration.

bind b split-window "tmux lsw | percol --initial-index $(tmux lsw | awk '/active.$/ {print NR-1}') | cut -d':' -f 1 | tr -d '\n' | xargs -0 tmux select-window -t"
bind B split-window "tmux ls | percol --initial-index $(tmux ls | awk \"/^$(tmux display-message -p '#{session_name}'):/ {print NR-1}\") | cut -d':' -f 1 | tr -d '\n' | xargs -0 tmux switch-client -t"

By putting above 2 settings into tmux.conf, you can select a tmux window with ${TMUX_PREFIX} b keys and session with ${TMUX_PREFIX} B keys.

Attaching to running tmux sessions can also be made easier with percol with this function(tested to work in bash and zsh)

function pattach() {
    if [[ $1 == "" ]]; then
        PERCOL=percol
    else
        PERCOL="percol --query $1"
    fi

    sessions=$(tmux ls)
    [ $? -ne 0 ] && return

    session=$(echo $sessions | eval $PERCOL | cut -d : -f 1)
    if [[ -n "$session" ]]; then
        tmux att -t $session
    fi
}

Calling percol from Python

Even though Percol is mainly designed as a UNIX command line tool, you can call it from your Python code like so:

from cStringIO import StringIO
from percol import Percol
from percol.actions import no_output

def main(candidates):
    si, so, se = StringIO(), StringIO(), StringIO()
    with Percol(
            actions=[no_output],
            descriptors={'stdin': si, 'stdout': so, 'stderr': se},
            candidates=iter(candidates)) as p:
        p.loop()
    results = p.model_candidate.get_selected_results_with_index()
    return [r[0] for r in results]

if __name__ == "__main__":
    candidates = ['foo', 'bar', 'baz']
    results = main(candidates)
    print("You picked: {!r}".format(results))

Configuration

Configuration file for percol should be placed under ${HOME}/.percol.d/ and named rc.py.

Here is an example ~/.percol.d/rc.py.

# X / _ / X
percol.view.PROMPT  = ur"<bold><yellow>X / _ / X</yellow></bold> %q"

# Emacs like
percol.import_keymap({
    "C-h" : lambda percol: percol.command.delete_backward_char(),
    "C-d" : lambda percol: percol.command.delete_forward_char(),
    "C-k" : lambda percol: percol.command.kill_end_of_line(),
    "C-y" : lambda percol: percol.command.yank(),
    "C-t" : lambda percol: percol.command.transpose_chars(),
    "C-a" : lambda percol: percol.command.beginning_of_line(),
    "C-e" : lambda percol: percol.command.end_of_line(),
    "C-b" : lambda percol: percol.command.backward_char(),
    "C-f" : lambda percol: percol.command.forward_char(),
    "M-f" : lambda percol: percol.command.forward_word(),
    "M-b" : lambda percol: percol.command.backward_word(),
    "M-d" : lambda percol: percol.command.delete_forward_word(),
    "M-h" : lambda percol: percol.command.delete_backward_word(),
    "C-n" : lambda percol: percol.command.select_next(),
    "C-p" : lambda percol: percol.command.select_previous(),
    "C-v" : lambda percol: percol.command.select_next_page(),
    "M-v" : lambda percol: percol.command.select_previous_page(),
    "M-<" : lambda percol: percol.command.select_top(),
    "M->" : lambda percol: percol.command.select_bottom(),
    "C-m" : lambda percol: percol.finish(),
    "C-j" : lambda percol: percol.finish(),
    "C-g" : lambda percol: percol.cancel(),
})

Customizing prompt

In percol, a prompt consists of two part: PROMPT and RPROMPT, like zsh. As the following example shows, each part appearance can be customized by specifying a prompt format into percol.view.PROMPT and percol.view.RPROMPT respectively.

percol.view.PROMPT = ur"<blue>Input:</blue> %q"
percol.view.RPROMPT = ur"(%F) [%i/%I]"

In prompt formats, a character preceded by % indicates a prompt format specifier and is expanded into a corresponding system value.

%%
- Display % itself
%q
- Display query and caret
%Q
- Display query without caret
%n
- Page number
%N
- Total page number
%i
- Current line number
%I
- Total line number
%c
- Caret position
%k
- Last input key

Dynamic prompt

By changing percol.view.PROMPT into a getter, percol prompts becomes more fancy.

# Change prompt in response to the status of case sensitivity
percol.view.__class__.PROMPT = property(
    lambda self:
    ur"<bold><blue>QUERY </blue>[a]:</bold> %q" if percol.model.finder.case_insensitive
    else ur"<bold><green>QUERY </green>[A]:</bold> %q"
)

Custom format specifiers

# Display finder name in RPROMPT
percol.view.prompt_replacees["F"] = lambda self, **args: self.model.finder.get_name()
percol.view.RPROMPT = ur"(%F) [%i/%I]"

Customizing styles

For now, styles of following 4 items can be customized in rc.py.

percol.view.CANDIDATES_LINE_BASIC    = ("on_default", "default")
percol.view.CANDIDATES_LINE_SELECTED = ("underline", "on_yellow", "white")
percol.view.CANDIDATES_LINE_MARKED   = ("bold", "on_cyan", "black")
percol.view.CANDIDATES_LINE_QUERY    = ("yellow", "bold")

Each RHS is a tuple of style specifiers listed below.

Foreground Colors

"black" for curses.COLOR_BLACK
"red" for curses.COLOR_RED
"green" for curses.COLOR_GREEN
"yellow" for curses.COLOR_YELLOW
"blue" for curses.COLOR_BLUE
"magenta" for curses.COLOR_MAGENTA
"cyan" for curses.COLOR_CYAN
"white" for curses.COLOR_WHITE

Background Color

"on_black" for curses.COLOR_BLACK
"on_red" for curses.COLOR_RED
"on_green" for curses.COLOR_GREEN
"on_yellow" for curses.COLOR_YELLOW
"on_blue" for curses.COLOR_BLUE
"on_magenta" for curses.COLOR_MAGENTA
"on_cyan" for curses.COLOR_CYAN
"on_white" for curses.COLOR_WHITE

Attributes

"altcharset" for curses.A_ALTCHARSET
"blink" for curses.A_BLINK
"bold" for curses.A_BOLD
"dim" for curses.A_DIM
"normal" for curses.A_NORMAL
"standout" for curses.A_STANDOUT
"underline" for curses.A_UNDERLINE
"reverse" for curses.A_REVERSE

Matching Method

By default, percol interprets input queries by users as string. If you prefer regular expression, try --match-method command line option.

$ percol --match-method regex

Migemo support

percol supports migemo (http://0xcc.net/migemo/) matching, which allows us to search Japanese documents with ASCII characters.

$ percol --match-method migemo

To use this feature, you need to install C/Migemo (https://github.com/koron/cmigemo). In Ubuntu, it's simple:

$ sudo apt-get install cmigemo

After that, by specifying a command line argument --match-method migemo, you can use migemo in percol.

NOTE: This feature uses python-cmigemo package (https://github.com/mooz/python-cmigemo). Doing pip install percol also installs this package too.

Dictionary settings

By default, percol assumes the path of a dictionary for migemo is /usr/local/share/migemo/utf-8/migemo-dict. If the dictionary is located in a different place, you should tell the location via rc.py.

For example, if the path of the dictionary is /path/to/a/migemo-dict, put lines below into your rc.py.

from percol.finder import FinderMultiQueryMigemo
FinderMultiQueryMigemo.dictionary_path = "/path/to/a/migemo-dict"

Minimum query length

If the query length is too short, migemo generates very long regular expression. To deal with this problem, percol does not pass a query if the length of the query is shorter than 2 and treat the query as raw regular expression.

To change this behavior, change the value of FinderMultiQueryMigemo.minimum_query_length like following settings.

from percol.finder import FinderMultiQueryMigemo
FinderMultiQueryMigemo.minimum_query_length = 1

Pinyin support

Now percol supports pinyin (http://en.wikipedia.org/wiki/Pinyin) for matching Chinese characters.

$ percol --match-method pinyin

In this matching method, first char of each Chinese character's pinyin sequence is used for matching. For example, 'zw' matches '中文' (ZhongWen), '中午'(ZhongWu), '作为' (ZuoWei) etc.

Extra package pinin(https://pypi.python.org/pypi/pinyin/0.2.5) needed.

Switching matching method dynamically

Matching method can be switched dynamically (at run time) by executing percol.command.specify_finder(FinderClass) or percol.command.toggle_finder(FinderClass). In addition, percol.command.specify_case_sensitive(case_sensitive) and percol.command.toggle_case_sensitive() change the matching status of case sensitivity.

from percol.finder import FinderMultiQueryMigemo, FinderMultiQueryRegex
percol.import_keymap({
    "M-c" : lambda percol: percol.command.toggle_case_sensitive(),
    "M-m" : lambda percol: percol.command.toggle_finder(FinderMultiQueryMigemo),
    "M-r" : lambda percol: percol.command.toggle_finder(FinderMultiQueryRegex)
})

Tips

Selecting multiple candidates

You can select and let percol to output multiple candidates by percol.command.toggle_mark_and_next() (which is bound to C-SPC by default).

percol.command.mark_all(), percol.command.unmark_all() and percol.command.toggle_mark_all() are useful to mark / unmark all candidates at once.

Z Shell support

A zsh completing-function for percol is available in https://github.com/mooz/percol/blob/master/tools/zsh/_percol .

Comments

make releases

Hi!

I'm looking at packaging this for Debian, and for that purpose it would be useful to have release numbers. Otherwise I'd just package it as 0~20140716 (for example).

Thanks

opened by anarcat 4
Spaces in search query
Is it somehow possible to have search queries contain spaces, because whenever I have a space in a query, it seems to interpret it as two separate queries.

For example (in regex mode): [0-9]{2} USER (i.e. a space, two numbers, a space and then the string USER) should match

CHAPTER 26 USER: 180 CHAPTER 26 USER: 2

But also matches:

BOOK 6 USER: 787 BOOK 6 USER: 89 LOCATION 111 USER: 709 LOCATION 120 USER: 2

Is there anyway to tell Percol to interpret the spaces as part of the search query?
opened by troeggla 3
Using `percol` from `fish` shell

Trying to use percol from fish as output to other commands doesn't work.

ls | percol works as expected.

cd (ls | percol) (not a typo, fish uses (...) instead of bash/zsh's $(...)) closes without doing anything.

opened by ghost 3
Prompt at the bottom
I'd like to have prompt at the bottom because you need to change your focus to the top of the terminal when start using percol. It would be nice if percol can show candidates and prompt "upside down", like this:

| | | candidate 3 | candidate 2 | candidate 1 | QUERY>

Can percol have an option to do this?
opened by tkf 3
Add feature to dynamically change case sensitive and match method

I've implemented a feature to toggle options in the same percol session.

Because I'm a begginer of Python, would you like to review my code? If you like this feature, please pull the commit.

Sincerely, kbkbkbkb1

opened by kai2nenobu 3

locale.Error: unsupported locale setting

Hi I run percol right after installing but I got an error. So I looked in the code and found the reason why I got this error. Anybody else got the same error?

Traceback (most recent call last):
  File "/usr/local/bin/percol", line 14, in <module>
    main()
  File "/usr/local/lib/python2.7/site-packages/percol/cli.py", line 186, in main
    output_encoding = set_proper_locale(options)
  File "/usr/local/lib/python2.7/site-packages/percol/cli.py", line 122, in set_proper_locale
    locale.setlocale(locale.LC_ALL, '')
  File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/locale.py", line 581,in setlocale
    return _setlocale(category, locale)
locale.Error: unsupported locale setting

I fixed this proble by just putting specific locale "C" into the second parameter. @line 122

locale.setlocale(locale.LC_ALL, 'C')

opened by dyong0 2

Problem binding C-j and RET
When I rebind C-j, the binding for RET is also changed. When I bind RET, neither binding is changed.

REPRODUCE CASE 1: Put this in ~/.percol.d/rc.py:

percol.import_keymap({ "C-j" : lambda percol: percol.command.select_next(), })

While using percol, press RET

Expected result: selected entry is printed to stdout (default RET binding) Actual result: the next entry becomes selected

REPRODUCE CASE 2: Put this in ~/.percol.d/rc.py:

percol.import_keymap({ "RET" : lambda percol: percol.command.select_next(), })

While using percol, press RET

Expected result: the next entry becomes selected Actual result: selected entry is printed to stdout (default C-j binding)
opened by guiniol 2

Incorrect fsf address

Hello.

I've package percol for Fedora and discower you have outdated file headers:

percol.x86_64: E: incorrect-fsf-address /usr/bin/percol
percol.x86_64: E: incorrect-fsf-address /usr/lib/python3.4/site-packages/percol/command.py
percol.x86_64: E: incorrect-fsf-address /usr/lib/python3.4/site-packages/percol/debug.py
percol.x86_64: E: incorrect-fsf-address /usr/lib/python3.4/site-packages/percol/view.py
percol.x86_64: E: incorrect-fsf-address /usr/lib/python3.4/site-packages/percol/tty.py
percol.x86_64: E: incorrect-fsf-address /usr/lib/python3.4/site-packages/percol/ansi.py
percol.x86_64: E: incorrect-fsf-address /usr/lib/python3.4/site-packages/percol/markup.py
percol.x86_64: E: incorrect-fsf-address /usr/lib/python3.4/site-packages/percol/display.py
percol.x86_64: E: incorrect-fsf-address /usr/lib/python3.4/site-packages/percol/action.py
percol.x86_64: E: incorrect-fsf-address /usr/lib/python3.4/site-packages/percol/lazyarray.py
percol.x86_64: E: incorrect-fsf-address /usr/lib/python3.4/site-packages/percol/__init__.py
percol.x86_64: E: incorrect-fsf-address /usr/lib/python3.4/site-packages/percol/info.py
percol.x86_64: E: incorrect-fsf-address /usr/lib/python3.4/site-packages/percol/model.py
percol.x86_64: E: incorrect-fsf-address /usr/lib/python3.4/site-packages/percol/actions.py
percol.x86_64: E: incorrect-fsf-address /usr/lib/python3.4/site-packages/percol/cli.py
percol.x86_64: E: incorrect-fsf-address /usr/lib/python3.4/site-packages/percol/key.py
percol.x86_64: E: incorrect-fsf-address /usr/lib/python3.4/site-packages/percol/finder.py

By our guidelines https://fedoraproject.org/wiki/Common_Rpmlint_issues#incorrect-fsf-address I shoul inform you.

opened by Hubbitus 2

--auto-match drops 2nd item in list
Hello,

I found an interesting bug when using the --auto-match flag. I can reproduce it with 0.1.0 installed from pypi on linux and osx. I am using python 2.7.

First, the correct behavior, without --auto-match:

$ echo -e "1\n2\n3\n4\n5" | percol QUERY> (1/5) [1/1] 1 2 3 4 5

Next, the buggy behavior with --auto-match, the 2nd list item is dropped:

$ echo -e "1\n2\n3\n4\n5" | percol --auto-match QUERY> (1/4) [1/1] 1 3 4 5

Interesting, huh? :smile_cat:

If I check out the original commit where this feature was added, 00b5417f, then I get the correct behavior. If I check out the commit where this feature was refactored, bd59b87d, then I get the buggy behavior. So perhaps that refactoring is now doing something unsafe (consuming the 2nd element somehow).
opened by lost-theory 2
Request: Customizable colors

I like that Percol colorizes text and the selected line background, but the white-on-magenta of the selected line is not very readable on my screen, because the contrast between the text and the magenta background is not very strong. After some digging I found where I can change the colors in view.py, but maintaining my own fork for this seems like overkill.

It'd be really nice if Percol supported customizable colors, perhaps by command-line options, or by a config file.

Thanks a lot for Percol!

opened by alphapapa 2
Allow running as a symlink.
If the percol executable is a symlink, abspath returns the directory containing the link, while realpath returns the directory containing the target. So now I can create

local/bin/percol -> local/src/percol/bin/percol

and I don't need local/src/percol/bin in my $PATH.
opened by ChickenProp 2
Percol marked as broken in Nix

I've been getting an environment stood up using nixpkgs. To my delight I found that I can install packages that normally are part of a language runtime (such as percol). percol even has a Nix package! However it is marked as broken as of writing. This typically stops a nix install of this package or any of its dependents.

The nix file mentions "missing cmigemo package which is missing libmigemo.so" as the direct reason for being marked as broken. I am unfamiliar with the Python ecosystem as well as Nix so I can't really speak to any of that. I'm mostly sharing awareness here and possibly hoping that maybe the information is dated :)

It also mentions that percol doesn't support Python 3. It's not clear if that's a requirement to no longer be marked as broken. I see there's some tickets in the issue tracker about that. I would think with Nix this would be fine conceptually, but I don't see anything in the package that jumps out as demanding Python 2.x (again, I'm unfamiliar with both ecosystems here).

For what it's worth, there is an allowBroken = true value that Nix consumers can set in their configuration to allow the installation to go forward. My limited reading on the topic suggests that the flag is kind of the equivalent of saying the tests don't pass for the package in question.

Thanks for maintaining a very helpful program!

opened by LoganBarnett 0

Example code in readme does not work with latest PyPI percol 0.2.1

Hi!

I would like to vote for for a new release. The readme changes merged from 4b28037e328da3d0fe8165c11b800cbaddcb525e are not supported: Once I fix the sting IO import for Python 3 (#107) I get ImportError: cannot import name 'no_output' from 'percol.actions' for a good reason: It does not exist in release 0.2.1. Can we have a new release please please?

If anyone needs a version of the example that works with Python 3.7:

from io import StringIO
from percol import Percol
from percol.actions import action

@action()
def no_output(lines, percol):
    "ignore all output"
    pass

def main(candidates):
    si, so, se = StringIO(), StringIO(), StringIO()
    with Percol(
            actions=[no_output],
            descriptors={'stdin': si, 'stdout': so, 'stderr': se},
            candidates=iter(candidates)) as p:
        p.loop()
    if p.args_for_action is None:
        raise KeyboardInterrupt
    results = p.model_candidate.get_selected_results_with_index()
    return [r[0] for r in results]

if __name__ == "__main__":
    candidates = ['foo', 'bar', 'baz']
    results = main(candidates)
    print("You picked: {!r}".format(results))

Best, Sebastian

opened by hartwork 0

python 3 support?

The readme says that only python 2 is supported. Is that still the case?

Googling "python 3 percol" revieled a bunch of clones in the while, allegedly with support for python 3.

opened by plainas 1

Releases(v0.1.0)

v0.1.0(Mar 18, 2015)
Do pip install percol -U to upgrade to the latest release.

Supports Python 3

Add a match method 'pinin'

Add an option -v, which inverts matches as in grep

Now percol displays runtime errors to screen

Use python-cmigemo for C/Migemo feature (Frees users from installing the extra package C/Migemo native Python extension)

Source code(tar.gz)
Source code(zip)