Adds a dockerfile. https://www.docker.com/ Docker basically lets you configure a development environment as code so no one needs to install python/pip/conda on their computers. I wanted to add this in to encourage more people to contribute, it should let people get up and running in 2 minutes.

Building the image:

docker build -t eht-imaging .

Running the image:

docker run -v "$(pwd)":/eht-imaging -it eht-imaging:latest bash

At this point you're ready to develop or can cd into ./examples and run python example.py.

Hope this isn't aggressive in any way and I'm not explaining something you all already know but linting makes code more maintainable and easier for a community to get behind by adhering to certain stylistic conventions and can help catch bugs as well as make things very readable for people familiar with the language.

https://en.wikipedia.org/wiki/Lint_(software)

There's a project called autopep8 that automatically runs pylint and fixes the formatting errors.

pip install --upgrade autopep8 installs it.

I wrote this command to go through and automatically update every .py file.

find . -type f -name '*.py' -exec autopep8 --in-place --aggressive --aggressive {} \;

Two functions added:

• load_obs(): takes in any observation file (or in the case of MAPS, multiple files) and uses the extension and input arguments to determine how to load the file. For uvfits, txt, and oifits files, extension is used and passed to the corresponding function. For MAPS, it checks for two additional arguments, obsspec and ifile. The function has knowledge of all the arguments for load_uvfits(), load_txt(), load_oifits(), and load_maps(), and the arguments can be passed to load_obs() as if you were using the legacy function calls. Example usage: ehtim.obsdata.load_obs("./sample_data.uvfits"), ehtim.obsdata.load_obs("./sample_data.txt", polrep='stokes'). Tested on txt and uvfits files, but I couldn't get my hands on any oifits or MAPS files quickly.
• getClosestScan(): returns the scan of an observation closest in time to the argument. Uses obs.split_obs() to get a list of scans for an observation, and then checks each scan's tstart value to figure out which is closest in time to the argument. If you want to save time, you can preload the output of split_obs to getClosestScan() as an argument and it will skip generating it. Time is input in GMST. Example usage: obs.getClosestScan(4) and obs.getClosestScan(4, splitObs=obs.split_obs()). Tested on sample data file in data/.

Cool picture, guys! But it seems like you've chosen a non-perceptually uniform colormap, which means that humans are likely to slightly misinterpret the image. It'd be great to see this historic image in something more human-comprehensible, like viridis or plasma rather than afmhot!

I'm calling the imager_func like this: zbl = 0.28 cfloor = 0.0028 niter = 300 conv_criteria = 0.0001 out = eh.imager_func( obs, gaussprior, gaussprior, zbl, d1='cphase', d2='camp', s1='gs', s2='gs', alpha_d1=50, alpha_d2=50, clipfloor=cfloor, maxit=niter, stop=conv_criteria )

The gaussprior is a single gaussian.

I get the following error and traceback:

/home/lmorabit/.local/lib/python2.7/site-packages/ehtim/imaging/imager_utils.pyc in imager_func(Obsdata, InitIm, Prior, flux, d1, d2, d3, alpha_d1, alpha_d2, alpha_d3, s1, s2, s3, alpha_s1, alpha_s2, alpha_s3, alpha_flux, alpha_cm, **kwargs) 294 # Print stats 295 print("Initial S_1: %f S_2: %f S_3: %f" % (reg1(ninit), reg2(ninit), reg3(ninit))) --> 296 print("Initial Chi^2_1: %f Chi^2_2: %f Chi^2_3: %f" % (chisq1(ninit), chisq2(ninit), chisq3(ninit))) 297 print("Initial Objective Function: %f" % (objfunc(xinit))) 298

/home/lmorabit/.local/lib/python2.7/site-packages/ehtim/imaging/imager_utils.pyc in chisq2(imvec) 199 200 def chisq2(imvec): --> 201 return chisq(imvec, A2, data2, sigma2, d2, ttype=ttype, mask=embed_mask) 202 203 def chisq2grad(imvec):

/home/lmorabit/.local/lib/python2.7/site-packages/ehtim/imaging/imager_utils.pyc in chisq(imvec, A, data, sigma, dtype, ttype, mask) 373 chisq = chisq_cphase(imvec, A, data, sigma) 374 elif dtype == 'camp': --> 375 chisq = chisq_camp(imvec, A, data, sigma) 376 elif dtype == 'logcamp': 377 chisq = chisq_logcamp(imvec, A, data, sigma)

/home/lmorabit/.local/lib/python2.7/site-packages/ehtim/imaging/imager_utils.pyc in chisq_camp(imvec, Amatrices, clamp, sigma) 739 def chisq_camp(imvec, Amatrices, clamp, sigma): 740 """Closure Amplitudes (normalized) chi-squared""" --> 741 print( np.min(imvec) ) 742 print( np.min(Amatrices[1] ) ) 743 clamp_samples = np.abs(np.dot(Amatrices[0], imvec) * np.dot(Amatrices[1], imvec) / (np.dot(Amatrices[2], imvec) * np.dot(Amatrices[3], imvec)))

/usr/lib64/python2.7/site-packages/numpy/core/fromnumeric.pyc in amin(a, axis, out, keepdims) 2370 2371 return _methods._amin(a, axis=axis, -> 2372 out=out, **kwargs) 2373 2374

/usr/lib64/python2.7/site-packages/numpy/core/_methods.pyc in _amin(a, axis, out, keepdims) 27 28 def _amin(a, axis=None, out=None, keepdims=False): ---> 29 return umr_minimum(a, axis, None, out, keepdims) 30 31 def _sum(a, axis=None, dtype=None, out=None, keepdims=False):

ValueError: zero-size array to reduction operation minimum which has no identity

• The requirements.txt is now installable by pip install -r requirements.txt.
  • Pip couldn't figure out how to install pynfft (due to external dependency) and pyfits (due to depending on an old setuptools).
• Removed unnecessary dependencies (pyfits, html).
• Added environment.yml. This gives a pinned set of packages on which eht-imaging works.
• Updated Dockerfile to Python 3.x.

This migrates the docker image to the python 3

Python has the html package included in the standard library so I also changed the requirements.txt file to only install html when using python 2

Sorry to open this as an issue, but I really wanted to say congrats to your whole team who made it possible to achieve something truly incredible. I really wish people could admire work like this a billion times than all the celebrities and stuff they usually admire. Brilliant work and kudos to your complete team!

@chanchikwan @lindyblackburn @klbouman @danielpalumbo @jrfarah

:DDDDDDD

Hi,

I am attempting to make some models for the international baselines of LOFAR using the EHT imager but I have stumbled on an error with the imaging function I do not quite understand.

This is how I call it:

eh.imager_func( obs, gaussprior, gaussprior, zbl, d1='cphase', d2='camp', s1='gs', s2='gs', alpha_d1=50, alpha_d2=50, clipfloor=0.001, maxit=300, stop=0.0001 )

Obs is generated from a measurement set (which was turned into uvfits).

Sorry if this is obscure but it looks like a dimensional issue. Let me know if you need extra information. Hopefully this is a simple issue and a mistake on my behalf.

Thanks!

Joe

Getting bispectra:: type vis, count min, scan 3370/3370 

/home/callingham/.local/lib/python2.7/site-packages/ehtim/imaging/imager_utils.py:2251: RuntimeWarning: invalid value encountered in greater
  snrmask = np.abs(clphasearr['cphase']/clphasearr['sigmacp']) > snrcut
Traceback (most recent call last):
  File "eht_imaging_joe.py", line 371, in <module>
    main( args.vis, closure_tels = args.ctels, npix = args.npix, fov_arcsec = args.fov, maxarcmin = args.maxarcmin, zbl = args.zbl, prior_fwhm_arcsec = args.prior_fwhm, doplots = args.doplots, imfile = args.imfile, remove_tels = args.rtels, niter=args.maxiter, cfloor=args.clipfloor, conv_criteria=args.convg, use_bs=args.use_bs, scratch_model=args.scratch_model, do_diagnostic_plots=args.diagplots, lotss_file=args.lotss_file )
  File "eht_imaging_joe.py", line 320, in main
    out = eh.imager_func( obs, gaussprior, gaussprior, zbl, d1='cphase', d2='camp', s1='gs', s2='gs', alpha_d1=50, alpha_d2=50, clipfloor=cfloor, maxit=niter, stop=conv_criteria )
  File "/home/callingham/.local/lib/python2.7/site-packages/ehtim/imaging/imager_utils.py", line 190, in imager_func
    (data1, sigma1, A1) = chisqdata(Obsdata, Prior, embed_mask, d1, **kwargs)
  File "/home/callingham/.local/lib/python2.7/site-packages/ehtim/imaging/imager_utils.py", line 625, in chisqdata
    (data, sigma, A) = chisqdata_cphase(Obsdata, Prior, mask, **kwargs)
  File "/home/callingham/.local/lib/python2.7/site-packages/ehtim/imaging/imager_utils.py", line 2259, in chisqdata_cphase
    sigma = np.linalg.norm([sigma, systematic_cphase_noise*np.ones(len(sigma))], axis=0)[snrmask]
IndexError: boolean index did not match indexed array along dimension 0; dimension is 70628 but corresponding boolean dimension is 70631
Exception TypeError: TypeError("'NoneType' object is not callable",) in <bound method UmfpackContext.new_del of <scipy.sparse.linalg.dsolve.umfpack.umfpack.UmfpackContext object at 0x4f39910>> ignored

I tried running example.py line by line in a Python terminal but it fails. It throws an error specifically on line 76 of example.py:

out = eh.imager_func(obs, gaussprior, gaussprior, flux, d1 = 'bs', s1 = 'simple', alpha_s1 = 1, alpha_d1 = 100, alpha_flux = 100, alpha_cm = 50, maxit = 100)

Here's the traceback:

Getting bispectra: type: vis count: min scan 107/107 Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/sean/eht/eht-imaging/ehtim/imaging/imager_utils.py", line 178, in imager_func
    (data1, sigma1, A1) = chisqdata(Obsdata, Prior, embed_mask, d1, **kwargs)#ttype=ttype, fft_pad_factor=fft_pad_factor, conv_func=grid_conv, p_rad=grid_prad, order=fft_interp, debias=debias,snrcut=snrcut)
  File "/home/sean/eht/eht-imaging/ehtim/imaging/imager_utils.py", line 583, in chisqdata
    (data, sigma, A) = chisqdata_bs(Obsdata, Prior, mask, **kwargs)
  File "/home/sean/eht/eht-imaging/ehtim/imaging/imager_utils.py", line 1884, in chisqdata_bs
    snrmask = np.abs(biarr['bispec']/data_arr['sigmab']) > snrcut
NameError: name 'data_arr' is not defined

Where should I define the global name data_arr for it to work?

There were two instances where a getMeasurementTerms function call used argument tot_flux=lightcurve[t], despite lightcurve=None, so when running starwarps, I would get the error: "NoneType not subscriptable."

I fixed this by just adding if statements.

conda install -c conda-forge pynfft will not work for newer M1 Mac / M1 Pro / M1 Max

I have forked the pyNFFT git repository here https://github.com/rohandahale/pyNFFT.git and I have written down step-by-step instructions for installing pyNFFT on M1 Macs. I have also made small edit to the setup.py file to use gcc-12 as the C compiler which works the best for installation with openmp.

It could be helpful if this repository link is added as a small note for pynfft installation in the README.md.

This line seems not to be necessary - the import isn't used and I successfully used the library without it. If removed, eht-imaging will no longer require the installation of the jupyter libraries, which would be nice.

https://github.com/achael/eht-imaging/blob/bbd3aeb06bef52bf89fa1c06de71e5509a5b0015/ehtim/modeling/modeling_utils.py#L49

:eyes: Some source code analysis tools can help to find opportunities for improving software components. :thought_balloon: I propose to increase the usage of augmented assignment statements accordingly.

Would you like to integrate anything from a transformation result which can be generated by a command like the following? (:point_right: Please check also for questionable change suggestions because of an evolving search pattern.)

lokal$ perl -p -i.orig -0777 -e 's/^(?<indentation>\s+)(?<target>\S+)\s*=\s*\k<target>[ \t]*(?<operator>[+%^@]|-(?!>)|&(?!&)|\|(?!\|)|\*\*?|\/\/?|<<|>>)/$+{indentation}$+{target} $+{operator}=/gm' $(find ~/Projekte/eht-imaging/lokal -name '*.py')

The reorder_baselines method of the Obsdata class assumes that any observations obtained on telescopes (t1, t2) and (t2, t1) are conjugates of each other, and throws away one of the two. However, this is not generally they case, in particular if the observations were obtained at different times and/or frequencies.

I discovered this when importing an OIFITS file, only to find that of the 60 or so wavelengths in the file, only a single one remains after reorder_baselines is run.

Describe the bug Program says that no NFFT is installed after following installation instructions in the README.

To Reproduce Steps to reproduce the behavior: 1.Install NFFT and pynfft with conda as described in readme 2. Clone repository 3. Build with dockerfile 3. Run dockerfile 4. Run first 35 lines of examples/examples.py in ipython

Expected behavior Running line 35 causes an error. I ran on both linux and MacOS; in the first case I got an issue that said no NFFT is installed, in the second I got an error that says 'NFFT is not defined.'

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information): macOS Catalina linux centOS Fedora

Describe the bug Building the docker file fails with error

To Reproduce Steps to reproduce the behavior:

1. Go to project root
2. Run docker build -t eht .
3. Scroll down to step [5/5] RUN pip install -r requirements.txt && conda install -y -c conda-forge pynfft && echo "backend: Agg" >> /opt/conda/lib/python2.7/site-packages/matplotlib/mpl-data/matplotlibrc
4. See error

Expected behavior The docker build should finish successfully

Screenshots

Desktop (please complete the following information):

• OS: MacOS Big Sur
• Docker Engine Version v19.03.13