A PyTorch Implementation of Neural IMage Assessment

https://github.com/kentsyx/Neural-IMage-Assessment/blob/9c50b3e384a88a8afdc00333d01656be5526bfed/model.py#L36

The EMD loss still seems to be wrong, my opinion is the sum operation should be inside of torch.abs

Originally posted by @luqiang360 in https://github.com/kentsyx/Neural-IMage-Assessment/issues/4#issuecomment-456781249

Should be the L2 distance between CDF of two distributions but not between the PDF of two distributions

And there's some typo in the naming such as emb and emd

Bumps pillow from 7.1.2 to 8.1.1.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Hi, @kentsyx @George3d6

I met this error when run python3 main.py

/home/tezro/.local/lib/python3.7/site-packages/torchvision/transforms/transforms.py:187: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead. warnings.warn("The use of the transforms.Scale transform is deprecated, " + Trainable params: 14.97 million /home/tezro/cocoapi/PythonAPI/Neural-IMage-Assessment/data_loader.py:31: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead. annotations = self.annotations.iloc[idx, 1:].as_matrix() /home/tezro/cocoapi/PythonAPI/Neural-IMage-Assessment/data_loader.py:31: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead. annotations = self.annotations.iloc[idx, 1:].as_matrix() /home/tezro/cocoapi/PythonAPI/Neural-IMage-Assessment/data_loader.py:31: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead. annotations = self.annotations.iloc[idx, 1:].as_matrix() /home/tezro/cocoapi/PythonAPI/Neural-IMage-Assessment/data_loader.py:31: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead. annotations = self.annotations.iloc[idx, 1:].as_matrix() Traceback (most recent call last): File "main.py", line 241, in main(config) File "main.py", line 96, in main for i, data in enumerate(train_loader): File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py", line 819, in next return self._process_data(data) File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py", line 846, in _process_data data.reraise() File "/usr/local/lib/python3.7/dist-packages/torch/_utils.py", line 385, in reraise raise self.exc_type(msg) RuntimeError: Caught RuntimeError in DataLoader worker process 0. Original Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop data = fetcher.fetch(index) File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/fetch.py", line 47, in fetch return self.collate_fn(data) File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/collate.py", line 74, in default_collate return {key: default_collate([d[key] for d in batch]) for key in elem} File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/collate.py", line 74, in return {key: default_collate([d[key] for d in batch]) for key in elem} File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/collate.py", line 55, in default_collate return torch.stack(batch, 0, out=out) RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 3 and 1 in dimension 1 at /pytorch/aten/src/TH/generic/THTensor.cpp:689

My System: Ubuntu 19.04, Pytorch 1.4, Torchvision 0.4.2, TitanXP.

Thanks in advance. Best from @bemoregt.

At line [66](https://github.com/kentsyx/Neural-IMage-Assessment/blob/f0028cd27de5cdb20a21c2b896999b3505bcb4f6/test.py#L66), it should be l+1 rather than l, because AVA votes start from 1 not 0.

I am trying to implement this for a single image and not getting any mean value below 5.0. The good quality images also at times return low values.

I am sharing the main.py file, please check if anything is wrong with the code.

import argparse
import os

import numpy as np
import matplotlib
import matplotlib.pyplot as plt

import torch
from torch import no_grad
import torch.autograd as autograd
import torch.optim as optim

import torchvision.transforms as transforms
import torchvision.datasets as dsets
import torchvision.models as models

import torch.nn.functional as F

from model import *

import cv2
file_name = 'bad'
filename = '/home/shayan/Projects/NIMA/images/'+file_name+'.jpg'

image = cv2.imread(filename)
image = cv2.resize(image,(224,224))

img_arr = image.transpose(2, 0, 1) # C x H x W
img_arr = np.expand_dims(img_arr,axis = 0)
print(img_arr.shape)

img_tensor = torch.from_numpy(img_arr)
img_tensor = img_tensor.type('torch.FloatTensor')
print(img_tensor.shape,img_tensor.size)

cuda = torch.cuda.is_available()
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
if cuda:
    print("Device: GPU")
else:
    print("Device: CPU")
    
base_model = models.vgg16(pretrained=True)
model = NIMA(base_model)

model.load_state_dict(torch.load("/home/shayan/Projects/NIMA/epoch-12.pkl", map_location=lambda storage, loc: storage))
print("Successfully loaded model")

with torch.no_grad():

    model.eval()

output = model(img_tensor)
output = output.view(10, 1)

predicted_mean, predicted_std = 0.0, 0.0
for i, elem in enumerate(output, 1):
    predicted_mean += i * elem
for j, elem in enumerate(output, 1):
    predicted_std += elem * (j - predicted_mean) ** 2
print("________________")
print(u"({}) \u00B1{}".format(round(float(predicted_mean),2), round(float(predicted_std), 2)))  

In line 195-196 of the main.py file you are computing the standard deviation of the score. The for loop is over the variable j while the variable used inside the loop is i. I just wanted to clarify if this is correct. Thank you in advance.

Flake8 testing of https://github.com/kentsyx/Neural-IMage-Assessment on Python 3.6.4

$ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics

./model.py:40:14: F821 undefined name 'xrange'
    for i in xrange(1, length + 1):
             ^
./model.py:57:14: F821 undefined name 'xrange'
    for i in xrange(mini_batch_size):
             ^
2     F821 undefined name 'xrange'
2

Hi Thanks for your code I really appreciate that but when I run the code using mac terminal, it keeps asking me for packages not installed, and I keep installing and installing but I still unable to run the training, and now I am sticking with NoModelNamed tensorboardX,

Is there basic solid steps I should follow to have all the required packages then I can run the code?

Sincerely

Abdullah

Thank you for your excellent job. when I run the test.py with the csv file ,it seems wrong in the following line: "gt = test_df[test_df[0] == img].to_numpy()[:, 1:].reshape(10, 1)", some value errors occurred as following: ValueError: cannot reshape array of size 0 into shape (10,1). So sorry to disturb you about this issue. Looking forward to your help.

Maybe the format of the csv is different from that of the paper, How to generate the specified 11 columns csv file of the paper ?

I would like to use your pre-training model, but the link to Google Cloud Drive is not working, I would like to request you to fill in the pre-training model, thank you very much.