I'm using nbdime-1.0.1.

http://localhost:8888/nbdime/git-difftool?base=fastai%2Fcourses%2Fdl1%2Flesson1.ipynb

Hitting: Hide unchanged cells

still shows all the code cells, changed and unchanged. The only difference is the the right column where the cell is unchanged is empty. This seems to be a bug, as it's not hiding the unchanged cells.

Perhaps the issue is that In[\d\d] is different in the original and the modified notebook. Yet if I don't hit 'Hide Unchanged cells' it doesn't display them as changed - and show to the full screen width.

Yet, it does report/hide some unchanged cells properly:

So, something is incomplete.

both snapshots are from the same diff.

Thank you!

Updates dependancies to Jupyterlab 3.0 rc6 and preemptively increments a major version to allow for testing with jupyterlab-git extension, sister PR at https://github.com/jupyterlab/jupyterlab-git/pull/818

Marked as draft until 3.0 release when this PR will be updated.

@vidartf I may be jumping the gun here, but is there a basic dockerfile available that will install and then run the nbdime webapp.

I manually installed a dev version, following this and then tried running python nbdime/webapp/nbdimeserver.py -p 8888, but wasn't able to get anything running. Maybe this is still a WIP.

I was experimenting around getting a number of 'services' running under jupyterhub, using kubernetes and wanted to add nbdime as a service in order to potentially be able to access its API. Is this viable to do, as yet?

Thanks.

Overview

This change adds a server endpoint to diff two git refs for a single file. This is flexible to perform diffs across various points in the Git lifecycle such as diff'ing changes in the Working Tree, Index, or two arbitrary commits. There are two reserved refs WORKING and INDEX to designate the Working Tree and the git Index

A separate API was added to avoid using the "git:" prefix to designate git diffs and add many optional parameters in the existing /api/diff. This is backwards compatible, the existing /api/diff will work as expected.

Addresses #480

curl -X POST \
  http://localhost:8888/nbdime/api/diff \
  -H 'Content-Type: application/json' \
  -H 'cache-control: no-cache' \
  -d '{
    "base": "git:Untitled1.ipynb"
}'

Other details

• Submitting early to get feedback. I'll add tests if the overall approach looks fine

Testing scenarios

Various cases for the git diff API

(Diff all changes in Working Tree)

curl -X POST \
  http://localhost:8888/nbdime/api/gitdiff \
  -H 'Content-Type: application/json' \
  -H 'cache-control: no-cache' \
  -d '{
	"ref_prev": "HEAD",
	"ref_curr": "WORKING",
	"file_name": "Untitled1.ipynb"
}'

(Diff all changes in staging)

curl -X POST \
  http://localhost:8888/nbdime/api/gitdiff \
  -H 'Content-Type: application/json' \
  -H 'cache-control: no-cache' \
  -d '{
	"ref_prev": "HEAD",
	"ref_curr": "INDEX",
	"file_name": "Untitled1.ipynb"
}'

(Diff changes between to Git refs in the local or remote history)

curl -X POST \
  http://localhost:8888/nbdime/api/gitdiff \
  -H 'Content-Type: application/json' \
  -H 'cache-control: no-cache' \
  -d '{
	"ref_prev": "origin/HEAD",
	"ref_curr": "HEAD",
	"file_name": "Untitled1.ipynb"
}'

Resolves #244 .

TODO:

• [x] make sure base_url is supported in every web application
• [x] prefix URLs with base_url in typescript
• [x] run tests with base URL, e.g. a%20b to ensure double-escapes don't happen

@martinal @minrk Opening this as a checklist for any doc updates prior to the next release. Please add or update. Happy to pitch in before after Thanksgiving holiday here.

• [x] Review installation instructions in Sphinx docs
• [x] Review installation instructions in README
• [x] Add any new features/functionality to Sphinx docs
• [x] Add a changelog doc (Initial release yay!)

I have nbdime configured to be used as a driver for 'git diff' and 'git merge':

nbdime config-git --enable --global

and I have nbdime configured to ignore all but code cells: (~/.jupyter/nbdime_config.json):

{

  "Extension": {
    "source": true,
    "details": false, 
    "outputs": false,
    "metadata": false
  },

  "NbDiff": {
    "source": true,
    "details": false, 
    "outputs": false,
    "metadata": false
  },

  "NbDiffDriver": {
    "source": true,
    "details": false, 
    "outputs": false,
    "metadata": false
  },

  "NbMergeDriver": {
    "source": true,
    "details": false, 
    "outputs": false,
    "metadata": false
  },

  "dummy": {}


}

so now when I do 'git diff' I get:

## modified /cells/4/source:
@@ -1,3 +1,4 @@
+#test
 from pathlib import Path

which is perfect.

Now, the notebook under git control has everything committed (outputs, metadata, etc.). I want git to ignore any changes in outputs, metadata and details and commit only changes in code cells, exactly as it's shown in 'git diff' (via nbdime configured to only diff code cells).

How can I commit just the code changes?

I tested what I have now and I get all changes committed, not just code cells.

One way nbdime could do that is to:

• extract code cells diff
• replace modified notebook with the clean checked out version before any changes
• apply the diff
• commit

but I don't know how to make nbdime to do that (or perhaps it does it already).

and I realize that the diff might not apply, if there are cells that were added and removed, so perhaps this is not an option at all.

edit: So we ended up using nbstripout to remove all but what's needed to be stored under git. Next is to figure out how to do automated merging a stripped out notebook under git with the local unstripped out notebook. I was thinking that perhaps nbmerge could mirror nbstripout's logic and be able to automatically merge the two, knowing which cells are important and need to be merged and which cells can be ignored.

Thank you.

Motivated by a need to communicate the merge results/conflicts to the mergetool, I would propose to have the merge algorithm build a structure which defines its decisions. A gist outlining a proposal for such a format can be found here: https://gist.github.com/vidartf/5ff461d224583772ff799732956d8f07

This decision structure could then be applied to the base directly to produce the merged file or sent to the merge tool.

Advantages include:

• Mergetool gets all the information it needs:
  • Which diffs are in conflict with each others.
  • Can easily show local/remote diffs aligned with merged output.
  • Can show the source of all difference in merged vs. base (i.e. color highlighting to say "this change comes from local, and this change comes from remote", which is good for context).
  • Has the original diffs easily accessible if the user wants to overrule/reset default decisions.
• In the merge routine, the decision making is separated from how those decisions are applied to the notebook (e.g. inserting conflict markers). This should also help compartmentalize the code for easier maintenance and unit testing.

Input would be very welcome on:

• Are there any cases where this will break down / cause inefficiencies?
• Should there be any changes to the proposed format for describing the decisions?

Originally reported in https://github.com/kynan/nbstripout/issues/44#issuecomment-412489658, there seems to be an issue getting the correct filter name from the config.

Traceback (most recent call last):
File "/Users/trtim/miniconda3/envs/nbdime/bin/nbdiff", line 11, in sys.exit(main())
File "/Users/trtim/miniconda3/envs/nbdime/lib/python3.6/site-packages/nbdime/nbdiffapp.py", line 128, in main return main_diff(arguments)
File "/Users/trtim/miniconda3/envs/nbdime/lib/python3.6/site-packages/nbdime/nbdiffapp.py", line 39, in main_diff for fbase, fremote in changed_notebooks(base, remote, paths):
File "/Users/trtim/miniconda3/envs/nbdime/lib/python3.6/site-packages/nbdime/gitfiles.py", line 160, in changed_notebooks entry.b_path, entry.b_blob, ref_remote, repo_dir)
File "/Users/trtim/miniconda3/envs/nbdime/lib/python3.6/site-packages/nbdime/gitfiles.py", line 110, in _get_diff_entry_stream ret = apply_possible_filter(path)
File "/Users/trtim/miniconda3/envs/nbdime/lib/python3.6/site-packages/nbdime/vcs/git/filter_integration.py", line 80, in apply_possible_filter stderr=STDOUT, shell=USE_SHELL
File "/Users/trtim/miniconda3/envs/nbdime/lib/python3.6/subprocess.py", line 336, in check_output **kwargs).stdout
File "/Users/trtim/miniconda3/envs/nbdime/lib/python3.6/subprocess.py", line 403, in run with Popen(*popenargs, **kwargs) as process:
File "/Users/trtim/miniconda3/envs/nbdime/lib/python3.6/subprocess.py", line 709, in __init__ restore_signals, start_new_session)
File "/Users/trtim/miniconda3/envs/nbdime/lib/python3.6/subprocess.py", line 1344, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '"/Users/trtim/miniconda3/envs/nbdime/bin/python3.6"': '"/Users/trtim/miniconda3/envs/nbdime/bin/python3.6"' 

@timtroendle What is the output of running git check-attr -z filter <notebook path here> for the notebook in question?

I've stumbled upon a very strange problem. When diff is shown via jupyter - the code cells in the diff can't be selected/copied to be pasted somewhere else. This defeats the purpose :(

I saved the html and reduced the problematic html to:

<div class="CodeMirror-scroll" tabindex="-1" draggable="true">

if draggable is set to false it works (the text is selectable/copyable), with true it doesn't.

everything else on the page can be selected/copied just fine.

I had to save the diff as html and to run:

perl -pi -e 's|draggable="true"|draggable="false"|' diff.html

and then I was able to extract the code diff I was after.

this is with nbdime-1.0.1 - same behavior with both firefox and chrome on ubuntu.

Thank you.

Hi,

Is it possible to ignore outputs (like with nbdiff --ignore-outputs) when using this tool with git diff (i.e. after doing nbdime config-git --enable --global) ?

I have tried a few options (like editing the git global config files), but haven't gotten anything to work yet.

BTW thanks for making this tool, it is dearly needed.

Resolves https://github.com/jupyter/nbdime/issues/645

Creates a random cookie_secret and passes it to the tornado application. I copied the secret generation from the jupyter_server code here: https://github.com/jupyter-server/jupyter_server/blob/a55bc587e9c343509e95bd93961e27e331882834/jupyter_server/serverapp.py#L1047

I tested this in a fresh environment after installing jupyter_server==1.23.3 and again in a fresh environment after installing jupyter_server==2.0.1 and confirmed it's working with both.

Note: With jupyter_server 2 there is a separate warning thrown unrelated to this change, but I'll note it here (could probably be a followup issue):

 UserWarning: The Tornado web application does not have an 'identity_provider' defined in its settings. In future releases of jupyter_server, this will be a required key for all subclasses of `JupyterHandler`. For an example, see the jupyter_server source code for how to add an identity provider to the tornado settings: https://github.com/jupyter-server/jupyter_server/blob/aa8fd8b3faf37466eeb99689d5555314c5bf6640/jupyter_server/serverapp.py#L253

Installing nbdime will automatically pull in the latest release of jupyter_server which is currently version 2.0.1. If you run nbdiff-web with jupyter_server>=2 you'll encounter the errors below and the web-ui won't render.

System information:

• OS: macOS Monterey (12.6)
• Platform: x86
• Python: 3.9.15 (via pyenv)
• nbdime: 3.1.1

Steps to reproduce:

# In a fresh environment
pip install nbdime ipython_genutils

nbdiff-web base-notebook.ipynb changed-notebook.ipynb

Workaround:

pip install jupyter_server==1.23.3 nbdime ipython_genutils

nbdiff-web base-notebook.ipynb changed-notebook.ipynb

Errors:

[I nbdimeserver:422] Listening on 127.0.0.1, port 64307
[I webutil:29] URL: http://127.0.0.1:64307/diff?base=basic-notebook-main.ipynb&remote=basic-notebook.ipynb
[W warnings:109] /Users/MYUSER/.local/share/virtualenvs/sample-notebooks-5G7S8U4h/lib/python3.9/site-packages/jupyter_server/base/handlers.py:234: UserWarning: The Tornado web application does not have an 'identity_provider' defined in its settings. In future releases of jupyter_server, this will be a required key for all subclasses of `JupyterHandler`. For an example, see the jupyter_server source code for how to add an identity provider to the tornado settings: https://github.com/jupyter-server/jupyter_server/blob/aa8fd8b3faf37466eeb99689d5555314c5bf6640/jupyter_server/serverapp.py#L253
  warnings.warn(

[E web:1798] Uncaught exception GET /diff?base=basic-notebook-main.ipynb&remote=basic-notebook.ipynb (127.0.0.1)
HTTPServerRequest(protocol='http', host='127.0.0.1:64307', method='GET', uri='/diff?base=basic-notebook-main.ipynb&remote=basic-notebook.ipynb', version='HTTP/1.1', remote_ip='127.0.0.1')
Traceback (most recent call last):
  File "/Users/MYUSER/.local/share/virtualenvs/sample-notebooks-5G7S8U4h/lib/python3.9/site-packages/tornado/web.py", line 1692, in _execute
    result = await result
  File "/Users/MYUSER/.local/share/virtualenvs/sample-notebooks-5G7S8U4h/lib/python3.9/site-packages/jupyter_server/base/handlers.py", line 602, in prepare
    _user = await _user
  File "/Users/MYUSER/.local/share/virtualenvs/sample-notebooks-5G7S8U4h/lib/python3.9/site-packages/jupyter_server/auth/identity.py", line 237, in _get_user
    _cookie_user = self.get_user_cookie(handler)
  File "/Users/MYUSER/.local/share/virtualenvs/sample-notebooks-5G7S8U4h/lib/python3.9/site-packages/jupyter_server/auth/identity.py", line 395, in get_user_cookie
    _user_cookie = handler.get_secure_cookie(
  File "/Users/MYUSER/.local/share/virtualenvs/sample-notebooks-5G7S8U4h/lib/python3.9/site-packages/tornado/web.py", line 773, in get_secure_cookie
    self.require_setting("cookie_secret", "secure cookies")
  File "/Users/MYUSER/.local/share/virtualenvs/sample-notebooks-5G7S8U4h/lib/python3.9/site-packages/tornado/web.py", line 1592, in require_setting
    raise Exception(
Exception: You must define the 'cookie_secret' setting in your application to use secure cookies
[E web:1221] Uncaught exception in write_error
Traceback (most recent call last):
  File "/Users/MYUSER/.local/share/virtualenvs/sample-notebooks-5G7S8U4h/lib/python3.9/site-packages/tornado/web.py", line 1692, in _execute
    result = await result
  File "/Users/MYUSER/.local/share/virtualenvs/sample-notebooks-5G7S8U4h/lib/python3.9/site-packages/jupyter_server/base/handlers.py", line 602, in prepare
    _user = await _user
  File "/Users/MYUSER/.local/share/virtualenvs/sample-notebooks-5G7S8U4h/lib/python3.9/site-packages/jupyter_server/auth/identity.py", line 237, in _get_user
    _cookie_user = self.get_user_cookie(handler)
  File "/Users/MYUSER/.local/share/virtualenvs/sample-notebooks-5G7S8U4h/lib/python3.9/site-packages/jupyter_server/auth/identity.py", line 395, in get_user_cookie
    _user_cookie = handler.get_secure_cookie(
  File "/Users/MYUSER/.local/share/virtualenvs/sample-notebooks-5G7S8U4h/lib/python3.9/site-packages/tornado/web.py", line 773, in get_secure_cookie
    self.require_setting("cookie_secret", "secure cookies")
  File "/Users/MYUSER/.local/share/virtualenvs/sample-notebooks-5G7S8U4h/lib/python3.9/site-packages/tornado/web.py", line 1592, in require_setting
    raise Exception(
Exception: You must define the 'cookie_secret' setting in your application to use secure cookies

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/MYUSER/.local/share/virtualenvs/sample-notebooks-5G7S8U4h/lib/python3.9/site-packages/tornado/web.py", line 1219, in send_error
    self.write_error(status_code, **kwargs)
  File "/Users/MYUSER/.local/share/virtualenvs/sample-notebooks-5G7S8U4h/lib/python3.9/site-packages/nbdime/webapp/nbdimeserver.py", line 72, in write_error
    return super(NbdimeHandler, self).write_error(status_code, **kwargs)
  File "/Users/MYUSER/.local/share/virtualenvs/sample-notebooks-5G7S8U4h/lib/python3.9/site-packages/jupyter_server/base/handlers.py", line 698, in write_error
    html = self.render_template("%s.html" % status_code, **ns)
  File "/Users/MYUSER/.local/share/virtualenvs/sample-notebooks-5G7S8U4h/lib/python3.9/site-packages/jupyter_server/base/handlers.py", line 624, in render_template
    ns.update(self.template_namespace)
  File "/Users/MYUSER/.local/share/virtualenvs/sample-notebooks-5G7S8U4h/lib/python3.9/site-packages/jupyter_server/base/handlers.py", line 634, in template_namespace
    logged_in=self.logged_in,
  File "/Users/MYUSER/.local/share/virtualenvs/sample-notebooks-5G7S8U4h/lib/python3.9/site-packages/jupyter_server/base/handlers.py", line 179, in logged_in
    user = self.current_user
  File "/Users/MYUSER/.local/share/virtualenvs/sample-notebooks-5G7S8U4h/lib/python3.9/site-packages/tornado/web.py", line 1345, in current_user
    self._current_user = self.get_current_user()
  File "/Users/MYUSER/.local/share/virtualenvs/sample-notebooks-5G7S8U4h/lib/python3.9/site-packages/jupyter_server/base/handlers.py", line 158, in get_current_user
    raise RuntimeError(msg)
RuntimeError: Calling `MainDiffHandler.get_current_user()` directly is deprecated in jupyter-server 2.0. Use `self.current_user` instead (works in all versions).
[E log:75] {
  "Host": "127.0.0.1:64307",
  "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8",
  "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:107.0) Gecko/20100101 Firefox/107.0"
}
[E log:76] 500 GET /diff?base=basic-notebook-main.ipynb&remote=basic-notebook.ipynb (@127.0.0.1) 7.53ms referer=None

Based on the user feedback we have created, it is not clear to users that they can change the output displays. We would love to switch the output selector from a hover display to an always present tabbed interface.

Happy to do the work again.

I have been working for a while on a branch, all of a sudden typescript started failing to build. I figured I had added something into the branch that made it fail, after failing for a long time to figure it out I started a new codespace and ran npm install then npm build:dev on the master branch of a fresh machine. This new setup also failed to compile typescript in the same way.

I believe that the master branch is broken for any fresh builds. Updating typescript seems to resolve the issue and I have opened a pr to do so #641 I did add a new tsconfig to that to tighten things up a bit, but it if is too much to review I am happy to split it out into a smaller pr just to get typescript update without the stricter config.

Here is the build error happening on a fresh machine build:

@gwincr11 ➜ /workspaces/nbdime (master) $ npm run build:dev

> @ build:dev /workspaces/nbdime
> tsc --build

node_modules/@types/babel__traverse/index.d.ts:68:50 - error TS1005: ']' expected.

68 export type ArrayKeys<T> = keyof { [P in keyof T as T[P] extends any[] ? P : never]: P };
                                                    ~~

node_modules/@types/babel__traverse/index.d.ts:68:53 - error TS1005: ';' expected.

68 export type ArrayKeys<T> = keyof { [P in keyof T as T[P] extends any[] ? P : never]: P };
                                                       ~

node_modules/@types/babel__traverse/index.d.ts:68:58 - error TS1005: ';' expected.

68 export type ArrayKeys<T> = keyof { [P in keyof T as T[P] extends any[] ? P : never]: P };
                                                            ~~~~~~~

node_modules/@types/babel__traverse/index.d.ts:68:70 - error TS1011: An element access expression should take an argument.

68 export type ArrayKeys<T> = keyof { [P in keyof T as T[P] extends any[] ? P : never]: P };
                                                                        

node_modules/@types/babel__traverse/index.d.ts:68:83 - error TS1005: ';' expected.

68 export type ArrayKeys<T> = keyof { [P in keyof T as T[P] extends any[] ? P : never]: P };
                                                                                     ~

node_modules/@types/babel__traverse/index.d.ts:68:84 - error TS1128: Declaration or statement expected.

68 export type ArrayKeys<T> = keyof { [P in keyof T as T[P] extends any[] ? P : never]: P };
                                                                                      ~

node_modules/@types/babel__traverse/index.d.ts:68:88 - error TS1128: Declaration or statement expected.

68 export type ArrayKeys<T> = keyof { [P in keyof T as T[P] extends any[] ? P : never]: P };
                                                                                          ~


Found 7 errors.

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! @ build:dev: `tsc --build`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the @ build:dev script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/codespace/.npm/_logs/2022-11-30T17_38_46_504Z-debug.log
@gwincr11 ➜ /workspaces/nbdime (master) $