This is the code for "HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields".

Google

Last update: Jan 2, 2023

Overview

HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields

This is the code for "HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields".

This codebase implements HyperNeRF using JAX, building on JaxNeRF.

Demo

We provide an easy-to-get-started demo using Google Colab!

These Colabs will allow you to train a basic version of our method using Cloud TPUs (or GPUs) on Google Colab.

Note that due to limited compute resources available, these are not the fully featured models and will train quite slowly and the quality will likely not be that great. If you would like to train a fully featured model, please refer to the instructions below on how to train on your own machine.

Description	Link
Process a video into a dataset
Train HyperNeRF
Render HyperNeRF Videos

Setup

The code can be run under any environment with Python 3.8 and above. (It may run with lower versions, but we have not tested it).

We recommend using Miniconda and setting up an environment:

conda create --name hypernerf python=3.8

Next, install the required packages:

pip install -r requirements.txt

Install the appropriate JAX distribution for your environment by following the instructions here. For example:

# For CUDA version 11.1
pip install --upgrade "jax[cuda111]" -f https://storage.googleapis.com/jax-releases/jax_releases.html

Training

After preparing a dataset, you can train a Nerfie by running:

export DATASET_PATH=/path/to/dataset
export EXPERIMENT_PATH=/path/to/save/experiment/to
python train.py \
    --base_folder $EXPERIMENT_PATH \
    --gin_bindings="data_dir='$DATASET_PATH'" \
    --gin_configs configs/test_local.gin

To plot telemetry to Tensorboard and render checkpoints on the fly, also launch an evaluation job by running:

python eval.py \
    --base_folder $EXPERIMENT_PATH \
    --gin_bindings="data_dir='$DATASET_PATH'" \
    --gin_configs configs/test_local.gin

The two jobs should use a mutually exclusive set of GPUs. This division allows the training job to run without having to stop for evaluation.

Configuration

We use Gin for configuration.
We provide a couple preset configurations.
Please refer to config.py for documentation on what each configuration does.
Preset configs:
- hypernerf_vrig_ds_2d.gin: The deformable surface configuration for the validation rig (novel-view synthesis) experiments.
- hypernerf_vrig_ap_2d.gin: The axis-aligned plane configuration for the validation rig (novel-view synthesis) experiments.
- hypernerf_interp_ds_2d.gin: The deformable surface configuration for the interpolation experiments.
- hypernerf_interp_ap_2d.gin: The axis-aligned plane configuration for the interpolation experiments.

Dataset

The dataset uses the same format as Nerfies.

Citing

If you find our work useful, please consider citing:

@article{park2021hypernerf
  author    = {Park, Keunhong and Sinha, Utkarsh and Hedman, Peter and Barron, Jonathan T. and Bouaziz, Sofien and Goldman, Dan B and Martin-Brualla, Ricardo and Seitz, Steven M.},
  title     = {HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields},
  journal   = {arXiv preprint arXiv:2106.13228},
  year      = {2021},
}

Comments

Some tips on collecting selfie video for hypernerf?

Hi, Keunhong!

I was wondering if you could provide some rules-of-thumb on how to collect good selfie video so that hypernerf can converge to its best global optima. (My first trial wasn't so much successful despite calibrating my camera extrinsics using background only w/ COLMAP)

Any tips from others successfully trained their own selfie videos are welcomed to comment!

opened by deepshwang 3
code to generate figures in notebooks file

Hi Keunhong,

This is really nice work and I am wondering if you have tested the code to generate figures in the notebooks file (like hypernerf_ap_ds_figure.ipynb). I run this code and seems cannot work. Could you give some hints on how to use these code in the notebooks file? Thank you so much!

opened by Heng14 2

ValueError: Non-hashable static arguments are not supported

When I run the training script, I encounter the problems as follow. By the way, it's ok when I run training script from nerfies with similar config.

train.py:302] Starting training
Traceback (most recent call last):
  File "/root/anaconda3/lib/python3.9/site-packages/jax/api_util.py", line 146, in argnums_partial_except
    hash(static_arg)
TypeError: unhashable type: 'NerfModel'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/bfs/sz/Research/hypernerf/train.py", line 370, in <module>
    app.run(main)
  File "/root/anaconda3/lib/python3.9/site-packages/absl/app.py", line 312, in run
    _run_main(main, args)
  File "/root/anaconda3/lib/python3.9/site-packages/absl/app.py", line 258, in _run_main
    sys.exit(main(argv))
  File "/bfs/sz/Research/hypernerf/train.py", line 330, in main
    state, stats, keys, model_out = ptrain_step(
  File "/root/anaconda3/lib/python3.9/site-packages/jax/_src/traceback_util.py", line 183, in reraise_with_filtered_traceback
    return fun(*args, **kwargs)
  File "/root/anaconda3/lib/python3.9/site-packages/jax/_src/api.py", line 1669, in f_pmapped
    out = pxla.xla_pmap(
  File "/root/anaconda3/lib/python3.9/site-packages/jax/core.py", line 1620, in bind
    return call_bind(self, fun, *args, **params)
  File "/root/anaconda3/lib/python3.9/site-packages/jax/core.py", line 1551, in call_bind
    outs = primitive.process(top_trace, fun, tracers, params)
  File "/root/anaconda3/lib/python3.9/site-packages/jax/core.py", line 1623, in process
    return trace.process_map(self, fun, tracers, params)
  File "/root/anaconda3/lib/python3.9/site-packages/jax/core.py", line 606, in process_call
    return primitive.impl(f, *tracers, **params)
  File "/root/anaconda3/lib/python3.9/site-packages/jax/interpreters/pxla.py", line 624, in xla_pmap_impl
    compiled_fun, fingerprint = parallel_callable(fun, backend, axis_name, axis_size,
  File "/root/anaconda3/lib/python3.9/site-packages/jax/linear_util.py", line 262, in memoized_fun
    ans = call(fun, *args)
  File "/root/anaconda3/lib/python3.9/site-packages/jax/interpreters/pxla.py", line 712, in parallel_callable
    jaxpr, out_sharded_avals, consts = pe.trace_to_jaxpr_final(
  File "/root/anaconda3/lib/python3.9/site-packages/jax/interpreters/partial_eval.py", line 1284, in trace_to_jaxpr_final
    jaxpr, out_avals, consts = trace_to_subjaxpr_dynamic(fun, main, in_avals)
  File "/root/anaconda3/lib/python3.9/site-packages/jax/interpreters/partial_eval.py", line 1262, in trace_to_subjaxpr_dynamic
    ans = fun.call_wrapped(*in_tracers)
  File "/root/anaconda3/lib/python3.9/site-packages/jax/linear_util.py", line 166, in call_wrapped
    ans = self.f(*args, **dict(self.params, **kwargs))
  File "/root/anaconda3/lib/python3.9/site-packages/jax/_src/traceback_util.py", line 183, in reraise_with_filtered_traceback
    return fun(*args, **kwargs)
  File "/root/anaconda3/lib/python3.9/site-packages/jax/_src/api.py", line 413, in cache_miss
    f, args = argnums_partial_except(f, static_argnums, args, allow_invalid=True)
  File "/root/anaconda3/lib/python3.9/site-packages/jax/api_util.py", line 148, in argnums_partial_except
    raise ValueError(
jax._src.traceback_util.UnfilteredStackTrace: ValueError: Non-hashable static arguments are not supported, as this can lead to unexpected cache-misses. Static argument (index 0) of type <class 'hypernerf.models.NerfModel'> for function train_step is non-hashable.

The stack trace below excludes JAX-internal frames.
The preceding is the original exception that occurred, unmodified.

--------------------

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/bfs/sz/Research/hypernerf/train.py", line 370, in <module>
    app.run(main)
  File "/root/anaconda3/lib/python3.9/site-packages/absl/app.py", line 312, in run
    _run_main(main, args)
  File "/root/anaconda3/lib/python3.9/site-packages/absl/app.py", line 258, in _run_main
    sys.exit(main(argv))
  File "/bfs/sz/Research/hypernerf/train.py", line 330, in main
    state, stats, keys, model_out = ptrain_step(
  File "/root/anaconda3/lib/python3.9/site-packages/jax/api_util.py", line 148, in argnums_partial_except
    raise ValueError(
ValueError: Non-hashable static arguments are not supported, as this can lead to unexpected cache-misses. Static argument (index 0) of type <class 'hypernerf.models.NerfModel'> for function train_step is non-hashable.

the jax version is as below:

jax                                0.2.17
jaxlib                             0.1.66+cuda110

opened by Spark001 1

Inconsistent shapes between value and initializer for parameter "kernel" in "/warp_field/trunk/hidden_0"

in running eval.py with "hypernerf_vrig_ap_2d.gin" configuration I got this error:

raise errors.ScopeParamShapeError(name, self.path_text, flax.errors.ScopeParamShapeError: Inconsistent shapes between value and initializer for parameter "kernel" in "/warp_field/trunk/hidden_0": (32, 128), (47, 128). (https://flax.readthedocs.io/en/latest/flax.errors.html#flax.errors.ScopeParamShapeError)

anyone has any idea?

opened by mah-asd 1
Bump tensorflow from 2.6.0 to 2.6.1
⚠️ Dependabot is rebasing this PR ⚠️

Rebasing might not happen immediately, so don't worry if this takes some time.

Note: if you make any changes to this PR yourself, they will take precedence over the rebase.

Bumps tensorflow from 2.6.0 to 2.6.1.

Release notes

Sourced from tensorflow's releases.

TensorFlow 2.6.1

Release 2.6.1

This release introduces several vulnerability fixes:

Fixes a code injection issue in saved_model_cli (CVE-2021-41228)

Fixes a vulnerability due to use of uninitialized value in Tensorflow (CVE-2021-41225)

Fixes a heap OOB in FusedBatchNorm kernels (CVE-2021-41223)

Fixes an arbitrary memory read in ImmutableConst (CVE-2021-41227)

Fixes a heap OOB in SparseBinCount (CVE-2021-41226)

Fixes a heap OOB in SparseFillEmptyRows (CVE-2021-41224)

Fixes a segfault due to negative splits in SplitV (CVE-2021-41222)

Fixes segfaults and vulnerabilities caused by accesses to invalid memory during shape inference in Cudnn* ops (CVE-2021-41221)

Fixes a null pointer exception when Exit node is not preceded by Enter op (CVE-2021-41217)

Fixes an integer division by 0 in tf.raw_ops.AllToAll (CVE-2021-41218)

Fixes a use after free and a memory leak in CollectiveReduceV2 (CVE-2021-41220)

Fixes an undefined behavior via nullptr reference binding in sparse matrix multiplication (CVE-2021-41219)

Fixes a heap buffer overflow in Transpose (CVE-2021-41216)

Prevents deadlocks arising from mutually recursive tf.function objects (CVE-2021-41213)

Fixes a null pointer exception in DeserializeSparse (CVE-2021-41215)

Fixes an undefined behavior arising from reference binding to nullptr in tf.ragged.cross (CVE-2021-41214)

Fixes a heap OOB read in tf.ragged.cross (CVE-2021-41212)

Fixes a heap OOB in shape inference for QuantizeV2 (CVE-2021-41211)

Fixes a heap OOB read in all tf.raw_ops.QuantizeAndDequantizeV* ops (CVE-2021-41205)

Fixes an FPE in ParallelConcat (CVE-2021-41207)

Fixes FPE issues in convolutions with zero size filters (CVE-2021-41209)

Fixes a heap OOB read in tf.raw_ops.SparseCountSparseOutput (CVE-2021-41210)

Fixes vulnerabilities caused by incomplete validation in boosted trees code (CVE-2021-41208)

Fixes vulnerabilities caused by incomplete validation of shapes in multiple TF ops (CVE-2021-41206)

Fixes a segfault produced while copying constant resource tensor (CVE-2021-41204)

Fixes a vulnerability caused by unitialized access in EinsumHelper::ParseEquation (CVE-2021-41201)

Fixes several vulnerabilities and segfaults caused by missing validation during checkpoint loading (CVE-2021-41203)

Fixes an overflow producing a crash in tf.range (CVE-2021-41202)

Fixes an overflow producing a crash in tf.image.resize when size is large (CVE-2021-41199)

Fixes an overflow producing a crash in tf.tile when tiling tensor is large (CVE-2021-41198)

Fixes a vulnerability produced due to incomplete validation in tf.summary.create_file_writer (CVE-2021-41200)

Fixes multiple crashes due to overflow and CHECK-fail in ops with large tensor shapes (CVE-2021-41197)

Fixes a crash in max_pool3d when size argument is 0 or negative (CVE-2021-41196)

Fixes a crash in tf.math.segment_* operations (CVE-2021-41195)

Updates curl to 7.78.0 to handle CVE-2021-22922, CVE-2021-22923, CVE-2021-22924, CVE-2021-22925, and CVE-2021-22926.

Changelog

Sourced from tensorflow's changelog.

Release 2.6.1

This release introduces several vulnerability fixes:

Fixes a code injection issue in saved_model_cli (CVE-2021-41228)

Fixes a vulnerability due to use of uninitialized value in Tensorflow (CVE-2021-41225)

Fixes a heap OOB in FusedBatchNorm kernels (CVE-2021-41223)

Fixes an arbitrary memory read in ImmutableConst (CVE-2021-41227)

Fixes a heap OOB in SparseBinCount (CVE-2021-41226)

Fixes a heap OOB in SparseFillEmptyRows (CVE-2021-41224)

Fixes a segfault due to negative splits in SplitV (CVE-2021-41222)

Fixes segfaults and vulnerabilities caused by accesses to invalid memory during shape inference in Cudnn* ops (CVE-2021-41221)

Fixes a null pointer exception when Exit node is not preceded by Enter op (CVE-2021-41217)

Fixes an integer division by 0 in tf.raw_ops.AllToAll (CVE-2021-41218)

Fixes a use after free and a memory leak in CollectiveReduceV2 (CVE-2021-41220)

Fixes an undefined behavior via nullptr reference binding in sparse matrix multiplication (CVE-2021-41219)

Fixes a heap buffer overflow in Transpose (CVE-2021-41216)

Prevents deadlocks arising from mutually recursive tf.function objects (CVE-2021-41213)

Fixes a null pointer exception in DeserializeSparse (CVE-2021-41215)

Fixes an undefined behavior arising from reference binding to nullptr in tf.ragged.cross (CVE-2021-41214)

Fixes a heap OOB read in tf.ragged.cross (CVE-2021-41212)

Fixes a heap OOB in shape inference for QuantizeV2 (CVE-2021-41211)

Fixes a heap OOB read in all tf.raw_ops.QuantizeAndDequantizeV* ops (CVE-2021-41205)

Fixes an FPE in ParallelConcat (CVE-2021-41207)

Fixes FPE issues in convolutions with zero size filters (CVE-2021-41209)

Fixes a heap OOB read in tf.raw_ops.SparseCountSparseOutput (CVE-2021-41210)

... (truncated)

Commits

3aa40c3 Merge pull request #52889 from tensorflow/fix-build-1-on-r2.6

1a97260 Upper bound tensorflow_estimator to match release

b03a4f1 Merge pull request #52874 from tensorflow-jenkins/relnotes-2.6.1-11115

1067732 Update RELEASE.md

7882f9c Merge pull request #52875 from tensorflow-jenkins/version-numbers-2.6.1-6784

bf0618d Update version numbers to 2.6.1

edf2a35 Insert release notes place-fill

ea77035 Merge pull request #52865 from tensorflow/fix-build-2-on-r2.6

c42e447 Merge pull request #52864 from tensorflow/fix-build-1-on-r2.6

f0258a5 Fix build on tensorflow/core/kernels/boosted_trees/stats_ops.cc

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 1
Questions about camera parameters and scene parameter in dataset
Hello, I really appreciate to this awesome work!!

To experiment on your dataset with pytorch framework, First, I''m trying to reimplemen nerfies data loader(HyperNeRF data loader) in pytorch.

However, I have some questions about camera parameters and scene parameters in dataset.

What does the scale, scene_to_metric and bbox in scene.json file?? What is differences between above scale parameter and 'scale' of images like 4x? ( scale of image vs scale of scene )

[ scene_scale is applied ]

For llff data format, we generate rays using this code.

[llff data format - get ray function]

I think that is the simple version of yours(ignore the distortion and assume that principal points is almost same to the center of image)

[ yours ]

But you undistorted to generate dirs using principal points, tangential distortion, radial_distortion, and pixel_aspect_ratio. Is there huge difference?

Are HyperNerf and Nerfies is implemented in NDC(Normalized Device Coordinate)? Usually many Nerf-based models have been implemented in NDC.

It will be really helpful for me if you answer this question!! Thanks for your awesome work again!

Best regards Dogyoon Lee
opened by dogyoonlee 1
Questions about camera coordinate and poses
I really appreciate to your awesome work!!

I have some questions about the camera pose matrix in camera folder's json files.

I'm reimplementing your paper on pytorch based nerf code.

However, I have confused about camera coordinate in nerfies dataset.

In load_llff.py in nerf pytorch code, they load the camera poses from given poses_bounds.npy file and correct their correct rotation matrix like follow codes.(poses matrix format is (3, 5, frame_num)

However, I could not find any correction part on your code when you generate the poses of dataset after loading them from given camera information json files.

I guess you used different coordinate system compared to original nerf(using llff dataset)

Then if I want to use your dataset as llff format and ray function, should I correct poses matrix like above code?

If you don't mind, could you explain the camera coordinate axis format of COLMAP results, NeRF(llff), and Nerfies(HyperNerf)??

for dynamic scene dataset(called nvidia-dataset), there is no undistortion part for camera pose like follow part of your code.

I wonder the necessity of above part because nerfies dataset images seems to not be distorted. and the dynamic scene dataset is pre-undistorted image?

Did you implement your code on NDC coordinate?? I wonder should I use the ndc_rays

I really hope your answer and appreciate to your great work again!!

Thank you.
opened by dogyoonlee 1
Pipeline of the data preprocess

Hi, thanks for your great work! And I'm trying to run hyperNeRF on my own data captured with a mobile phone. But after running COLMAP on those frames, I don't know how to get all those parameters in your data class. Could you please offer a brief guide or something to refer to? I worry that I can't get the best performance of your work without a correct data preprocess. Thanks!

opened by buaacyw 0
Bump tensorflow from 2.6.1 to 2.6.3
Bumps tensorflow from 2.6.1 to 2.6.3.

Release notes

Sourced from tensorflow's releases.

TensorFlow 2.6.3

Release 2.6.3

This releases introduces several vulnerability fixes:

Fixes a floating point division by 0 when executing convolution operators (CVE-2022-21725)

Fixes a heap OOB read in shape inference for ReverseSequence (CVE-2022-21728)

Fixes a heap OOB access in Dequantize (CVE-2022-21726)

Fixes an integer overflow in shape inference for Dequantize (CVE-2022-21727)

Fixes a heap OOB access in FractionalAvgPoolGrad (CVE-2022-21730)

Fixes an overflow and divide by zero in UnravelIndex (CVE-2022-21729)

Fixes a type confusion in shape inference for ConcatV2 (CVE-2022-21731)

Fixes an OOM in ThreadPoolHandle (CVE-2022-21732)

Fixes an OOM due to integer overflow in StringNGrams (CVE-2022-21733)

Fixes more issues caused by incomplete validation in boosted trees code (CVE-2021-41208)

Fixes an integer overflows in most sparse component-wise ops (CVE-2022-23567)

Fixes an integer overflows in AddManySparseToTensorsMap (CVE-2022-23568)

Fixes a number of CHECK-failures in MapStage (CVE-2022-21734)

Fixes a division by zero in FractionalMaxPool (CVE-2022-21735)

Fixes a number of CHECK-fails when building invalid/overflowing tensor shapes (CVE-2022-23569)

Fixes an undefined behavior in SparseTensorSliceDataset (CVE-2022-21736)

Fixes an assertion failure based denial of service via faulty bin count operations (CVE-2022-21737)

Fixes a reference binding to null pointer in QuantizedMaxPool (CVE-2022-21739)

Fixes an integer overflow leading to crash in SparseCountSparseOutput (CVE-2022-21738)

Fixes a heap overflow in SparseCountSparseOutput (CVE-2022-21740)

Fixes an FPE in BiasAndClamp in TFLite (CVE-2022-23557)

Fixes an FPE in depthwise convolutions in TFLite (CVE-2022-21741)

Fixes an integer overflow in TFLite array creation (CVE-2022-23558)

Fixes an integer overflow in TFLite (CVE-2022-23559)

Fixes a dangerous OOB write in TFLite (CVE-2022-23561)

Fixes a vulnerability leading to read and write outside of bounds in TFLite (CVE-2022-23560)

Fixes a set of vulnerabilities caused by using insecure temporary files (CVE-2022-23563)

Fixes an integer overflow in Range resulting in undefined behavior and OOM (CVE-2022-23562)

Fixes a vulnerability where missing validation causes tf.sparse.split to crash when axis is a tuple (CVE-2021-41206)

Fixes a CHECK-fail when decoding resource handles from proto (CVE-2022-23564)

Fixes a CHECK-fail with repeated AttrDef (CVE-2022-23565)

Fixes a heap OOB write in Grappler (CVE-2022-23566)

Fixes a CHECK-fail when decoding invalid tensors from proto (CVE-2022-23571)

Fixes a null-dereference when specializing tensor type (CVE-2022-23570)

Fixes a crash when type cannot be specialized (CVE-2022-23572)

Fixes a heap OOB read/write in SpecializeType (CVE-2022-23574)

Fixes an unitialized variable access in AssignOp (CVE-2022-23573)

Fixes an integer overflow in OpLevelCostEstimator::CalculateTensorSize (CVE-2022-23575)

Fixes an integer overflow in OpLevelCostEstimator::CalculateOutputSize (CVE-2022-23576)

Fixes a null dereference in GetInitOp (CVE-2022-23577)

Fixes a memory leak when a graph node is invalid (CVE-2022-23578)

Fixes an abort caused by allocating a vector that is too large (CVE-2022-23580)

Fixes multiple CHECK-failures during Grappler's IsSimplifiableReshape (CVE-2022-23581)

Fixes multiple CHECK-failures during Grappler's SafeToRemoveIdentity (CVE-2022-23579)

Fixes multiple CHECK-failures in TensorByteSize (CVE-2022-23582)

... (truncated)

Changelog

Sourced from tensorflow's changelog.

Release 2.6.3

This releases introduces several vulnerability fixes:

Fixes a floating point division by 0 when executing convolution operators (CVE-2022-21725)

Fixes a heap OOB read in shape inference for ReverseSequence (CVE-2022-21728)

Fixes a heap OOB access in Dequantize (CVE-2022-21726)

Fixes an integer overflow in shape inference for Dequantize (CVE-2022-21727)

Fixes a heap OOB access in FractionalAvgPoolGrad (CVE-2022-21730)

Fixes an overflow and divide by zero in UnravelIndex (CVE-2022-21729)

Fixes a type confusion in shape inference for ConcatV2 (CVE-2022-21731)

Fixes an OOM in ThreadPoolHandle (CVE-2022-21732)

Fixes an OOM due to integer overflow in StringNGrams (CVE-2022-21733)

Fixes more issues caused by incomplete validation in boosted trees code (CVE-2021-41208)

Fixes an integer overflows in most sparse component-wise ops (CVE-2022-23567)

Fixes an integer overflows in AddManySparseToTensorsMap (CVE-2022-23568)

Fixes a number of CHECK-failures in MapStage (CVE-2022-21734)

Fixes a division by zero in FractionalMaxPool (CVE-2022-21735)

Fixes a number of CHECK-fails when building invalid/overflowing tensor shapes (CVE-2022-23569)

Fixes an undefined behavior in SparseTensorSliceDataset (CVE-2022-21736)

Fixes an assertion failure based denial of service via faulty bin count operations (CVE-2022-21737)

Fixes a reference binding to null pointer in QuantizedMaxPool (CVE-2022-21739)

Fixes an integer overflow leading to crash in SparseCountSparseOutput (CVE-2022-21738)

Fixes a heap overflow in SparseCountSparseOutput (CVE-2022-21740)

Fixes an FPE in BiasAndClamp in TFLite (CVE-2022-23557)

Fixes an FPE in depthwise convolutions in TFLite (CVE-2022-21741)

... (truncated)

Commits

92a6bb0 Merge pull request #54204 from tensorflow/disable-flaky-tests-on-r2.6

3dc94ed Merge pull request #54203 from tensorflow/fix-sanity-on-r2.6

52f4b51 Disable flaky test

ad069c0 Reorder tags to fix buildifier linting

1db0b90 Merge pull request #54200 from tensorflow/cherrypick-510ae18200d0a4fad797c0bf...

dcbbb77 Set Env Variable to override Setuptools new behavior

fd16fa9 Merge pull request #54174 from tensorflow-jenkins/relnotes-2.6.3-32160

bc97b92 Update RELEASE.md

1e6335b Merge pull request #54184 from tensorflow/cherrypick-d437dec4d549fc30f9b85c75...

083c3c6 Update third_party/icu/workspace.bzl

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 0
Bump numpy from 1.19.5 to 1.21.0
Bumps numpy from 1.19.5 to 1.21.0.

Release notes

Sourced from numpy's releases.

v1.21.0

NumPy 1.21.0 Release Notes

The NumPy 1.21.0 release highlights are

continued SIMD work covering more functions and platforms,

initial work on the new dtype infrastructure and casting,

universal2 wheels for Python 3.8 and Python 3.9 on Mac,

improved documentation,

improved annotations,

new PCG64DXSM bitgenerator for random numbers.

In addition there are the usual large number of bug fixes and other improvements.

The Python versions supported for this release are 3.7-3.9. Official support for Python 3.10 will be added when it is released.

:warning: Warning: there are unresolved problems compiling NumPy 1.21.0 with gcc-11.1 .

Optimization level -O3 results in many wrong warnings when running the tests.

On some hardware NumPy will hang in an infinite loop.

New functions

Add PCG64DXSM BitGenerator

Uses of the PCG64 BitGenerator in a massively-parallel context have been shown to have statistical weaknesses that were not apparent at the first release in numpy 1.17. Most users will never observe this weakness and are safe to continue to use PCG64. We have introduced a new PCG64DXSM BitGenerator that will eventually become the new default BitGenerator implementation used by default_rng in future releases. PCG64DXSM solves the statistical weakness while preserving the performance and the features of PCG64.

See upgrading-pcg64 for more details.

(gh-18906)

Expired deprecations

The shape argument numpy.unravel_index cannot be passed as dims keyword argument anymore. (Was deprecated in NumPy 1.16.)

... (truncated)

Commits

b235f9e Merge pull request #19283 from charris/prepare-1.21.0-release

34aebc2 MAINT: Update 1.21.0-notes.rst

493b64b MAINT: Update 1.21.0-changelog.rst

07d7e72 MAINT: Remove accidentally created directory.

032fca5 Merge pull request #19280 from charris/backport-19277

7d25b81 BUG: Fix refcount leak in ResultType

fa5754e BUG: Add missing DECREF in new path

61127bb Merge pull request #19268 from charris/backport-19264

143d45f Merge pull request #19269 from charris/backport-19228

d80e473 BUG: Removed typing for == and != in dtypes

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 0
Bump pillow from 8.3.2 to 9.0.0
Bumps pillow from 8.3.2 to 9.0.0.

Release notes

Sourced from pillow's releases.

9.0.0

https://pillow.readthedocs.io/en/stable/releasenotes/9.0.0.html

Changes

Restrict builtins for ImageMath.eval() #5923 [@radarhere]

Ensure JpegImagePlugin stops at the end of a truncated file #5921 [@radarhere]

Fixed ImagePath.Path array handling #5920 [@radarhere]

Remove consecutive duplicate tiles that only differ by their offset #5919 [@radarhere]

Removed redundant part of condition #5915 [@radarhere]

Explicitly enable strip chopping for large uncompressed TIFFs #5517 [@kmilos]

Use the Windows method to get TCL functions on Cygwin #5807 [@DWesl]

Changed error type to allow for incremental WebP parsing #5404 [@radarhere]

Improved I;16 operations on big endian #5901 [@radarhere]

Ensure that BMP pixel data offset does not ignore palette #5899 [@radarhere]

Limit quantized palette to number of colors #5879 [@radarhere]

Use latin1 encoding to decode bytes #5870 [@radarhere]

Fixed palette index for zeroed color in FASTOCTREE quantize #5869 [@radarhere]

When saving RGBA to GIF, make use of first transparent palette entry #5859 [@radarhere]

Pass SAMPLEFORMAT to libtiff #5848 [@radarhere]

Added rounding when converting P and PA #5824 [@radarhere]

Improved putdata() documentation and data handling #5910 [@radarhere]

Exclude carriage return in PDF regex to help prevent ReDoS #5912 [@radarhere]

Image.NONE is only used for resampling and dithers #5908 [@radarhere]

Fixed freeing pointer in ImageDraw.Outline.transform #5909 [@radarhere]

Add Tidelift alignment action and badge #5763 [@aclark4life]

Replaced further direct invocations of setup.py #5906 [@radarhere]

Added ImageShow support for xdg-open #5897 [@m-shinder]

Fixed typo #5902 [@radarhere]

Switched from deprecated "setup.py install" to "pip install ." #5896 [@radarhere]

Support 16-bit grayscale ImageQt conversion #5856 [@cmbruns]

Fixed raising OSError in _safe_read when size is greater than SAFEBLOCK #5872 [@radarhere]

Convert subsequent GIF frames to RGB or RGBA #5857 [@radarhere]

WebP: Fix memory leak during decoding on failure #5798 [@ilai-deutel]

Do not prematurely return in ImageFile when saving to stdout #5665 [@infmagic2047]

Added support for top right and bottom right TGA orientations #5829 [@radarhere]

Corrected ICNS file length in header #5845 [@radarhere]

Block tile TIFF tags when saving #5839 [@radarhere]

Added line width argument to ImageDraw polygon #5694 [@radarhere]

Do not redeclare class each time when converting to NumPy #5844 [@radarhere]

Only prevent repeated polygon pixels when drawing with transparency #5835 [@radarhere]

Fix pushes_fd method signature #5833 [@hoodmane]

Add support for pickling TrueType fonts #5826 [@hugovk]

Only prefer command line tools SDK on macOS over default MacOSX SDK #5828 [@radarhere]

Fix compilation on 64-bit Termux #5793 [@landfillbaby]

Replace 'setup.py sdist' with '-m build --sdist' #5785 [@hugovk]

Use declarative package configuration #5784 [@hugovk]

Use title for display in ImageShow #5788 [@radarhere]

Fix for PyQt6 #5775 [@hugovk]

... (truncated)

Changelog

Sourced from pillow's changelog.

9.0.0 (2022-01-02)

Restrict builtins for ImageMath.eval(). CVE-2022-22817 #5923 [radarhere]

Ensure JpegImagePlugin stops at the end of a truncated file #5921 [radarhere]

Fixed ImagePath.Path array handling. CVE-2022-22815, CVE-2022-22816 #5920 [radarhere]

Remove consecutive duplicate tiles that only differ by their offset #5919 [radarhere]

Improved I;16 operations on big endian #5901 [radarhere]

Limit quantized palette to number of colors #5879 [radarhere]

Fixed palette index for zeroed color in FASTOCTREE quantize #5869 [radarhere]

When saving RGBA to GIF, make use of first transparent palette entry #5859 [radarhere]

Pass SAMPLEFORMAT to libtiff #5848 [radarhere]

Added rounding when converting P and PA #5824 [radarhere]

Improved putdata() documentation and data handling #5910 [radarhere]

Exclude carriage return in PDF regex to help prevent ReDoS #5912 [hugovk]

Fixed freeing pointer in ImageDraw.Outline.transform #5909 [radarhere]

Added ImageShow support for xdg-open #5897 [m-shinder, radarhere]

Support 16-bit grayscale ImageQt conversion #5856 [cmbruns, radarhere]

Convert subsequent GIF frames to RGB or RGBA #5857 [radarhere]

... (truncated)

Commits

82541b6 9.0.0 version bump

cae5ac4 Merge pull request #5924 from radarhere/cves

ed4cf78 CVEs TBD

d7f60d1 Merge pull request #5923 from radarhere/imagemath_eval

8531b01 Restrict builtins for ImageMath.eval

1efb1d9 Merge pull request #5922 from radarhere/releasenotes

f6c7871 Added release notes for #5919, #5920 and #5921

032d2dc Update CHANGES.rst [ci skip]

baae9ec Merge pull request #5921 from radarhere/jpeg_eoi

1059eb5 If appended EOI did not work, do not keep trying

Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR

@dependabot recreate will recreate this PR, overwriting any edits that have been made to it

@dependabot merge will merge this PR after your CI passes on it

@dependabot squash and merge will squash and merge this PR after your CI passes on it

@dependabot cancel merge will cancel a previously requested merge and block automerging

@dependabot reopen will reopen this PR if it is closed

@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually

@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)

@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot use these labels will set the current labels as the default for future PRs for this repo and language

@dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language

@dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language

@dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the Security Alerts page.

dependencies
opened by dependabot[bot] 0
using gpu memory but not computation

I use A6000 gpu installed with nvidia driver version 11.6 but I use nvidia/cuda:11.2.0. image where cudnn version is 8.1.1

I installed all the packages in requirements.txt

The training is proceeding but GPU uses only memory and doesn't compute. I think that computation might be proceeded in the cpu.

Also, I change 'train.py' like this,

def main(argv): jax.config.parse_flags_with_absl() physical_devices = tf.config.list_physical_devices('GPU') tf.config.experimental.set_visible_devices(physical_devices[2:], 'GPU')

So I uninstalled tensorflow and newly install tensorflow-gpu, but the result is same.

I need help.

opened by psm1000 0
how to generate mesh data from your model?

Like other nerf tools, can I get mesh data or texture data from your tool? I cannot find where the similar functions in your source code. Thanks for your help in advance.

opened by leepeter2019 0
tree_map() missing 1 required positional argument: 'tree'

TypeError Traceback (most recent call last) in 22 train_eval_iter = datasource.create_iterator( 23 shuffled(datasource.train_ids), batch_size=0, devices=devices) ---> 24 val_eval_iter = datasource.create_iterator( 25 shuffled(datasource.val_ids), batch_size=0, devices=devices)

/usr/local/lib/python3.8/site-packages/hypernerf/datasets/core.py in create_iterator(self, item_ids, batch_size, repeat, flatten, shuffle, prefetch_size, shuffle_buffer_size, devices) 386 devices: Optional[Sequence[Any]] = None): 387 """Creates a dataset iterator for JAX.""" --> 388 dataset = self.create_dataset( 389 item_ids, 390 flatten=flatten,

/usr/local/lib/python3.8/site-packages/hypernerf/datasets/core.py in create_dataset(self, item_ids, flatten, shuffle, row_shuffle_buffer_size, pixel_shuffle_buffer_size) 406 """Creates a tf.data.Dataset.""" 407 if self.preload: --> 408 return self._create_preloaded_dataset( 409 item_ids, flatten=flatten, shuffle=shuffle) 410 else:

/usr/local/lib/python3.8/site-packages/hypernerf/datasets/core.py in _create_preloaded_dataset(self, item_ids, flatten, shuffle) 434 data_list = utils.parallel_map(load_fn, item_ids) 435 data_list = [_camera_to_rays_fn(item) for item in data_list] ... --> 340 return tree_util.tree_multimap(lambda *x: np.stack(x), *list_of_pytrees) 341 342

TypeError: tree_map() missing 1 required positional argument: 'tree'

opened by lishulincug 0
how to rerender video just like Project Page showing
Thanks a lot for sharing the code! I want to know how to use your code to realize dynamic video training and rendering just like the video below. Now I can only render the free view video with the object still, just like the first video on your project page.

When I try to execute python train.py --base_folder $EXPERIMENT_PATH --gin_bindings="data_dir='$DATASET_PATH'" --gin_configs configs/hypernerf_interp_ds_2d.gin

I0925 16:29:32.415290 140117065011584 train.py:135] *** Starting experiment I0925 16:29:32.415382 140117065011584 train.py:139] *** Loading Gin configs from: ['configs/hypernerf_interp_ds_2d.gin'] I0925 16:29:32.430606 140117065011584 xla_bridge.py:236] Unable to initialize backend 'tpu_driver': NOT_FOUND: Unable to find driver in registry given worker: I0925 16:29:32.494084 140117065011584 xla_bridge.py:236] Unable to initialize backend 'tpu': INVALID_ARGUMENT: TpuPlatform is not available. I0925 16:29:32.494200 140117065011584 train.py:159] exp_dir = /data/TinyMe/chickchickenout I0925 16:29:32.494389 140117065011584 train.py:163] summary_dir = /data/TinyMe/chickchickenout/summaries/train I0925 16:29:32.494438 140117065011584 train.py:167] checkpoint_dir = /data/TinyMe/chickchickenout/checkpoints I0925 16:29:32.494497 140117065011584 train.py:171] Starting process 0. There are 1 processes. I0925 16:29:32.494542 140117065011584 train.py:173] Found 1 accelerator devices: [GpuDevice(id=0, process_index=0)]. I0925 16:29:32.494578 140117065011584 train.py:175] Found 1 total devices: [GpuDevice(id=0, process_index=0)]. I0925 16:29:32.698378 140117065011584 train.py:187] Creating datasource I0925 16:29:32.698573 140117065011584 interp.py:71] *** Loading dataset IDs from /data/TinyMe/chickchicken/dataset.json I0925 16:29:32.699014 140117065011584 core.py:237] Creating datasource of type InterpDataSource with use_appearance_id=True, use_camera_id=False, use_warp_id=True, use_depth=False, use_time=False, train_stride=1, val_stride=1 I0925 16:29:32.699630 140117065011584 train.py:200] Initializing models. 2022-09-25 16:29:45.751242: E external/org_tensorflow/tensorflow/compiler/xla/pjrt/pjrt_stream_executor_client.cc:2085] Execution of replica 0 failed: INTERNAL: Failed to load in-memory CUBIN: CUDA_ERROR_ILLEGAL_ADDRESS: an illegal memory access was encountered Traceback (most recent call last): File "train.py", line 370, in app.run(main) File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/absl/app.py", line 312, in run _run_main(main, args) File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/absl/app.py", line 258, in _run_main sys.exit(main(argv)) File "train.py", line 203, in main model, params['model'] = models.construct_nerf( File "/data/TinyMe/hypernerf/hypernerf/models.py", line 701, in construct_nerf params = model.init({ File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/flax/linen/module.py", line 998, in init _, v_out = self.init_with_output( File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/flax/linen/module.py", line 968, in init_with_output return self.apply( File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/flax/linen/module.py", line 936, in apply return apply( File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/flax/core/scope.py", line 687, in wrapper y = fn(root, *args, **kwargs) File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/flax/linen/module.py", line 1178, in scope_fn return fn(module.clone(parent=scope), *args, **kwargs) File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/flax/linen/module.py", line 275, in wrapped_module_method y = fun(self, *args, **kwargs) File "/data/TinyMe/hypernerf/hypernerf/models.py", line 631, in call z_vals, points = model_utils.sample_pdf( File "/data/TinyMe/hypernerf/hypernerf/model_utils.py", line 216, in sample_pdf z_samples = piecewise_constant_pdf(key, bins, weights, num_coarse_samples, File "/data/TinyMe/hypernerf/hypernerf/model_utils.py", line 184, in piecewise_constant_pdf bins_g0, bins_g1 = minmax(bins) File "/data/TinyMe/hypernerf/hypernerf/model_utils.py", line 179, in minmax x1 = jnp.min(jnp.where(~mask, x[..., None], x[..., -1:, None]), -2) File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/jax/_src/traceback_util.py", line 162, in reraise_with_filtered_traceback return fun(*args, **kwargs) File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/jax/_src/api.py", line 412, in cache_miss out_flat = xla.xla_call( File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/jax/core.py", line 1616, in bind return call_bind(self, fun, *args, **params) File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/jax/core.py", line 1607, in call_bind outs = primitive.process(top_trace, fun, tracers, params) File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/jax/core.py", line 1619, in process return trace.process_call(self, fun, tracers, params) File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/jax/core.py", line 615, in process_call return primitive.impl(f, *tracers, **params) File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/jax/interpreters/xla.py", line 623, in _xla_call_impl out = compiled_fun(*args) File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/jax/interpreters/xla.py", line 913, in _execute_compiled out_bufs = compiled.execute(input_bufs) jax._src.traceback_util.UnfilteredStackTrace: RuntimeError: INTERNAL: Failed to load in-memory CUBIN: CUDA_ERROR_ILLEGAL_ADDRESS: an illegal memory access was encountered

The stack trace below excludes JAX-internal frames. The preceding is the original exception that occurred, unmodified.

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "train.py", line 370, in app.run(main) File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/absl/app.py", line 312, in run _run_main(main, args) File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/absl/app.py", line 258, in _run_main sys.exit(main(argv)) File "train.py", line 203, in main model, params['model'] = models.construct_nerf( File "/data/TinyMe/hypernerf/hypernerf/models.py", line 701, in construct_nerf params = model.init({ File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/flax/linen/module.py", line 998, in init _, v_out = self.init_with_output( File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/flax/linen/module.py", line 968, in init_with_output return self.apply( File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/flax/linen/module.py", line 936, in apply return apply( File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/flax/core/scope.py", line 687, in wrapper y = fn(root, *args, **kwargs) File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/flax/linen/module.py", line 1178, in scope_fn return fn(module.clone(parent=scope), *args, **kwargs) File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/flax/linen/module.py", line 275, in wrapped_module_method y = fun(self, *args, **kwargs) File "/data/TinyMe/hypernerf/hypernerf/models.py", line 631, in call z_vals, points = model_utils.sample_pdf( File "/data/TinyMe/hypernerf/hypernerf/model_utils.py", line 216, in sample_pdf z_samples = piecewise_constant_pdf(key, bins, weights, num_coarse_samples, File "/data/TinyMe/hypernerf/hypernerf/model_utils.py", line 184, in piecewise_constant_pdf bins_g0, bins_g1 = minmax(bins) File "/data/TinyMe/hypernerf/hypernerf/model_utils.py", line 179, in minmax x1 = jnp.min(jnp.where(~mask, x[..., None], x[..., -1:, None]), -2) File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/site-packages/jax/interpreters/xla.py", line 913, in _execute_compiled out_bufs = compiled.execute(input_bufs) RuntimeError: INTERNAL: Failed to load in-memory CUBIN: CUDA_ERROR_ILLEGAL_ADDRESS: an illegal memory access was encountered 2022-09-25 16:29:45.849669: E external/org_tensorflow/tensorflow/stream_executor/cuda/cuda_driver.cc:1047] could not synchronize on CUDA context: CUDA_ERROR_ILLEGAL_ADDRESS: an illegal memory access was encountered :: *** Begin stack trace ***

_PyDict_DelItem_KnownHash _PyEval_EvalFrameDefault _PyEval_EvalCodeWithName _PyFunction_Vectorcall PyObject_CallFunctionObjArgs PyObject_ClearWeakRefs _PyModule_ClearDict PyImport_Cleanup Py_FinalizeEx Py_RunMain Py_BytesMain __libc_start_main

*** End stack trace ***

2022-09-25 16:29:45.849777: F external/org_tensorflow/tensorflow/compiler/xla/service/gpu/gpu_executable.cc:124] Check failed: pair.first->SynchronizeAllActivity() Fatal Python error: Aborted

Current thread 0x00007f6f8be31180 (most recent call first): File "/home/hy/miniconda3/envs/hypernerf/lib/python3.8/weakref.py", line 351 in remove Aborted (core dumped)

And in sdf_2d.ipynb, it seems that the definition of the hyper_encoder module is missing.

thanks for your helping!

https://user-images.githubusercontent.com/54303737/192090214-c22ff504-50c5-491c-8c7c-62436219591b.mp4
opened by LangR7 0
Problem with Geman-McClure loss implementation

I have been testing the Geman-McClure loss implemented in the repo, more specifically the model_utils.general_loss_with_squared_residual() with alpha = -2.

However, the output, as shown below, is very different from my expectation. (alpha = -2, scale = 0.1) Shouldn't it be like the quadratic shape when |x| < scale

opened by JokerYan 2