GRU+SVM+DROPOUT+LR-DECAY in TF

https://github.com/mpekalski/zalando/blob/master/GRU%2BSVM%2BDROPOUT%2BLR-DECAY.ipynb

100 epochs, test accuracy: 0.9841001033782959

No preprocessing.

I have downloaded the t10k-images-idx3-ubyte.gz from the readme provided download link, but I checked the md5sum value is 9fb629c4189551a2d022fa330f9573f3, it not the same as readme given, I have deleted it and redownload again for five times, it is wrong.

Someone commented about this issue on Reddit (pasted below) and I think you should seriously consider changing the name of the benchmark to something else while it's still early on.

MNIST stands for "Modified National Institute of Standards and Technology" and "National Institute of Standards and Technology" might not be too happy with their name being used. Call it something else. Especially when its an entirely new dataset and not a modification/extension of original NIST dataset.

No preprocessing. See source code for exact network config.

Fashion-MNIST test accuracy: 97.39 % Digit-MNIST test accuracy: 99.13 %

Source code: https://github.com/rfratila/Vulcan/blob/master/train_mnist_conv.py

Built with Lasagne and Theano

Hi Everybody,

in a recent publication of mine, we surveyed popular data sets and looked into finding mislabeled instances. For Fashion MNST, we found a large number of mislabeled / incorrect instances (i.e. where the automated cutting failed). See table 7 on the last page. It contains 64 instances, but we found a lot more. While it may or may not be dramatic for training, this may be disadvantageous for evaluation (since it skews accuracy scores).

If you're interested in fixing / looking into this, let me know.

Best Nicolas

I added some examples (taken directly from the paper) . The heading indicates the label (which is incorrect as far as i can see) and the instance number in the training set.

Hello there, these MLP results say in the author README that 90% is the Validation Performance, not the test.

MLP 256-128-64 | None | 0.900 | - | @lianghong |   -- | -- | -- | -- | -- | --

https://github.com/abelusha/MNIST-Fashion-CNN/blob/master/MNIST_Fashon_CNN_using_Keras.ipynb

Preprocessing : Normalization Result:

Keras based Architecture:

I have several experimental results with different activation function and learning rate with CNN architecture like this : C(3,32)-C(3,32)-P2-C(3,64)-C(3,64)-P2-FC64-FC64-S10

|Activation|Learning Rate|MNIST|Fashion MNIST| |---|---|---|---| |RELU|0.01|0.9874|0.9883| |RELU|0.001|0.9388|0.9368| |SELU|0.01|0.9871|0.9819| |SELU|0.001|0.9490|0.8202|

For now, my best results are 98.74% and 98.83% for MNIST and Fashion-MNIST, respectively. The train-val curve could be found in my repository : https://github.com/JMingKuo/fashion-mnist

The model details are as follows:

Preprocessing by calculating mean and std beforehand
Trained using Cross Entropy Loss and Adam Optimizer.

The layers in sequence are:

Convolutional layer with 6 feature maps of size 5 x 5
BatchNorm layer followed by ReLU activation.
Average Pooling layer of size 2 x 2.
Convolutional layer with 16 feature maps of size 5 x 5
BatchNorm layer followed by ReLU activation.
Average Pooling layer of size 2 x 2.
Fully connected layer of input size 400 and output size 120 followed by ReLU activation.
Fully connected layer of input size 120 and output size 84 followed by ReLU activation.
Fully connected layer of input size 84 and output size 10 .

Accuracy achieved on Fashion MNIST Test Dataset is 99.2 % Accuracy achieved on MNIST Test Dataset is 99.1% .

This network has been implemented in PyTorch. The code can be found here

@hanxiao I wonder what's the clustering performance of the state-of-the-art clustering algorithms on fashion-mnist. I tested my algorithm and got an accuracy of 0.59 and NMI of 0.63. Have you collected other clustering results? Thanks.

Hi, we achieve 95.99% top-1 accuracy with using WRN-28-10 on Fashion-MNIST using standard preprocessing (mean/std subtraction/division) and augmentation (random crops/horizontal flips).

When using WRN-28-10 + Random Erasing, it gives 96.35% top-1 accuracy.

The code will be available soon on github.

Hi everyone! I have a project with fashion mnist , and I want use Incremental learning with it, could you tell me please is it possible or not? Thank you!

import argparse
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim from torchvision import datasets, transforms

class ConvBlocck(nn.Module): def init(self, inchannel, outchannel, kernel_size=3, stride=1): super().init() self.conv = nn.Sequential( nn.Conv2d(inchannel, outchannel, 1, 1), nn.BatchNorm2d(outchannel), nn.GELU(), ) self.conv1 = nn.Sequential( nn.Conv2d(outchannel, outchannel, kernel_size=kernel_size, padding=kernel_size//2, stride=stride, groups=outchannel), nn.BatchNorm2d(outchannel), nn.GELU(), ) self.kernel_size = kernel_size self.stride = stride def forward(self, x): out = self.conv(x) out = out + self.conv1(out) return out

class Net(nn.Module): def init(self): super(Net, self).init() self.conv1 = ConvBlocck(1, 20, 5, 1) self.conv2 = ConvBlocck(20, 50, 5, 1) self.conv3 = nn.Sequential( ConvBlocck(50, 100, 5, 1), ConvBlocck(100, 100, 7, 1), ) self.conv4 = nn.Sequential( ConvBlocck(100, 200, 5, 1), ConvBlocck(200, 200, 5, 1), )

    self.fc1 = nn.Linear(200, 100)
    self.fc2 = nn.Linear(100, 10)

def forward(self, x):
    x = self.conv1(x)
    x = F.max_pool2d(x, 2, 2)
    x = self.conv2(x)
    x = F.max_pool2d(x, 2, 2)
    x = self.conv3(x)
    x = self.conv4(x)
    x = F.avg_pool2d(x, kernel_size=7, stride=1, padding=0)
    # import pdb; pdb.set_trace()
    x = x.view(-1, 200)
    x = F.relu(self.fc1(x))
    x = self.fc2(x)
    return F.log_softmax(x, dim=1)

def train(args, model, device, train_loader, optimizer, epoch): model.train() for batch_idx, (data, target) in enumerate(train_loader): data, target = data.to(device), target.to(device) optimizer.zero_grad() output = model(data) loss = F.nll_loss(output, target) loss.backward() optimizer.step() if batch_idx % args.log_interval == 0: print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format( epoch, batch_idx * len(data), len(train_loader.dataset), 100. * batch_idx / len(train_loader), loss.item()))

def test(args, model, device, test_loader): model.eval() test_loss = 0 correct = 0 with torch.no_grad(): for data, target in test_loader: data, target = data.to(device), target.to(device) output = model(data) test_loss += F.nll_loss(output, target, reduction='sum').item() # sum up batch loss pred = output.argmax(dim=1, keepdim=True) # get the index of the max log-probability correct += pred.eq(target.view_as(pred)).sum().item()

test_loss /= len(test_loader.dataset)

print('\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format(
    test_loss, correct, len(test_loader.dataset),
    100. * correct / len(test_loader.dataset)))

def main(): # Training settings parser = argparse.ArgumentParser(description='PyTorch MNIST Example') parser.add_argument('--batch-size', type=int, default=128, metavar='N', help='input batch size for training (default: 64)') parser.add_argument('--test-batch-size', type=int, default=1000, metavar='N', help='input batch size for testing (default: 1000)') parser.add_argument('--epochs', type=int, default=140, metavar='N', help='number of epochs to train (default: 10)') parser.add_argument('--lr', type=float, default=0.1, metavar='LR', help='learning rate (default: 0.01)') parser.add_argument('--momentum', type=float, default=0.9, metavar='M', help='SGD momentum (default: 0.5)') parser.add_argument('--no-cuda', action='store_true', default=False, help='disables CUDA training') parser.add_argument('--seed', type=int, default=1, metavar='S', help='random seed (default: 1)') parser.add_argument('--log-interval', type=int, default=10, metavar='N', help='how many batches to wait before logging training status') parser.add_argument('--save-model', action='store_true', default=False, help='For Saving the current Model') args = parser.parse_args() use_cuda = not args.no_cuda and torch.cuda.is_available() torch.manual_seed(args.seed) device = torch.device("cuda" if use_cuda else "cpu") kwargs = {'num_workers': 4, 'pin_memory': True} if use_cuda else {} train_loader = torch.utils.data.DataLoader( datasets.FashionMNIST('./fashionmnist_data/', train=True, download=False, transform=transforms.Compose([ transforms.RandomCrop(28, padding=4), transforms.RandomHorizontalFlip(), transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,)), # transforms.RandomErasing(p = 0.5, scale = (0, 0.4), ratio = (0.5, 2), value = 'random'), ])), batch_size=args.batch_size, shuffle=True, **kwargs) test_loader = torch.utils.data.DataLoader( datasets.FashionMNIST('./fashionmnist_data/', train=True, transform=transforms.Compose([ transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,)) ])), batch_size=args.test_batch_size, shuffle=True, **kwargs) model = Net().to(device) optimizer = optim.AdamW(model.parameters(), eps=1e-8, betas=(0.9, 0.99), lr=5e-4, weight_decay=5e-2) scheduler = optim.lr_scheduler.MultiStepLR(optimizer, milestones=[50, 120], gamma=0.1) for epoch in range(1, args.epochs + 1): train(args, model, device, train_loader, optimizer, epoch) test(args, model, device, test_loader) if (args.save_model): torch.save(model.state_dict(), "mnist_cnn.pt") if name == 'main': main()

`

just run with python -u main.py

some log

Test set: Average loss: 0.0810, Accuracy: 58119/60000 (97%)

Train Epoch: 138 [0/60000 (0%)] Loss: 0.103322 Train Epoch: 138 [1280/60000 (2%)] Loss: 0.124087 Train Epoch: 138 [2560/60000 (4%)] Loss: 0.105898 Train Epoch: 138 [3840/60000 (6%)] Loss: 0.114831 Train Epoch: 138 [5120/60000 (9%)] Loss: 0.055228 Train Epoch: 138 [6400/60000 (11%)] Loss: 0.057790 Train Epoch: 138 [7680/60000 (13%)] Loss: 0.077030 Train Epoch: 138 [8960/60000 (15%)] Loss: 0.104552 Train Epoch: 138 [10240/60000 (17%)] Loss: 0.098626 Train Epoch: 138 [11520/60000 (19%)] Loss: 0.095885 Train Epoch: 138 [12800/60000 (21%)] Loss: 0.066495 Train Epoch: 138 [14080/60000 (23%)] Loss: 0.053589 Train Epoch: 138 [15360/60000 (26%)] Loss: 0.092867 Train Epoch: 138 [16640/60000 (28%)] Loss: 0.116169 Train Epoch: 138 [17920/60000 (30%)] Loss: 0.107934 Train Epoch: 138 [19200/60000 (32%)] Loss: 0.116899 Train Epoch: 138 [20480/60000 (34%)] Loss: 0.095697 Train Epoch: 138 [21760/60000 (36%)] Loss: 0.112671 Train Epoch: 138 [23040/60000 (38%)] Loss: 0.075007 Train Epoch: 138 [24320/60000 (41%)] Loss: 0.083380 Train Epoch: 138 [25600/60000 (43%)] Loss: 0.136541 Train Epoch: 138 [26880/60000 (45%)] Loss: 0.098393 Train Epoch: 138 [28160/60000 (47%)] Loss: 0.156382 Train Epoch: 138 [29440/60000 (49%)] Loss: 0.120168 Train Epoch: 138 [30720/60000 (51%)] Loss: 0.102728 Train Epoch: 138 [32000/60000 (53%)] Loss: 0.093192 Train Epoch: 138 [33280/60000 (55%)] Loss: 0.067673 Train Epoch: 138 [34560/60000 (58%)] Loss: 0.118263 Train Epoch: 138 [35840/60000 (60%)] Loss: 0.063559 Train Epoch: 138 [37120/60000 (62%)] Loss: 0.107007 Train Epoch: 138 [38400/60000 (64%)] Loss: 0.097562 Train Epoch: 138 [39680/60000 (66%)] Loss: 0.067643 Train Epoch: 138 [40960/60000 (68%)] Loss: 0.119229 Train Epoch: 138 [42240/60000 (70%)] Loss: 0.153711 Train Epoch: 138 [43520/60000 (72%)] Loss: 0.103719 Train Epoch: 138 [44800/60000 (75%)] Loss: 0.120675 Train Epoch: 138 [46080/60000 (77%)] Loss: 0.092273 Train Epoch: 138 [47360/60000 (79%)] Loss: 0.148049 Train Epoch: 138 [48640/60000 (81%)] Loss: 0.096311 Train Epoch: 138 [49920/60000 (83%)] Loss: 0.067373 Train Epoch: 138 [51200/60000 (85%)] Loss: 0.084663 Train Epoch: 138 [52480/60000 (87%)] Loss: 0.149150 Train Epoch: 138 [53760/60000 (90%)] Loss: 0.069273 Train Epoch: 138 [55040/60000 (92%)] Loss: 0.050591 Train Epoch: 138 [56320/60000 (94%)] Loss: 0.059370 Train Epoch: 138 [57600/60000 (96%)] Loss: 0.132310 Train Epoch: 138 [58880/60000 (98%)] Loss: 0.084755

Test set: Average loss: 0.0648, Accuracy: 58591/60000 (98%)

I downloaded the dataset from a source and placing it in an arbitrary path. And I found some people having trouble in loading the dataset from a local path using tensorflow 2.0. The API tf.keras.datasets.fashion_mnist.load_data() seems not support loading data locally.

I write a new function that may help to solve this issue. I hope that this function could help somebody in need. I don`t know whether the issue is big enough for a pull request. So I open an issue here and post my code here. Hope that I won't cause any inconvenience.

The code of new function:

import os
import numpy as np
import gzip
def load_data_fromlocalpath(input_path):
  """Loads the Fashion-MNIST dataset.
  Modified by Henry Huang in 2020/12/24.
  We assume that the input_path should in a correct path address format.
  We also assume that potential users put all the four files in the path.

  Load local data from path ‘input_path’.

  Returns:
      Tuple of Numpy arrays: `(x_train, y_train), (x_test, y_test)`.
  """
  files = [
      'train-labels-idx1-ubyte.gz', 'train-images-idx3-ubyte.gz',
      't10k-labels-idx1-ubyte.gz', 't10k-images-idx3-ubyte.gz'
  ]

  paths = []
  for fname in files:
    paths.append(os.path.join(input_path, fname))  # The location of the dataset.


  with gzip.open(paths[0], 'rb') as lbpath:
    y_train = np.frombuffer(lbpath.read(), np.uint8, offset=8)

  with gzip.open(paths[1], 'rb') as imgpath:
    x_train = np.frombuffer(
        imgpath.read(), np.uint8, offset=16).reshape(len(y_train), 28, 28)

  with gzip.open(paths[2], 'rb') as lbpath:
    y_test = np.frombuffer(lbpath.read(), np.uint8, offset=8)

  with gzip.open(paths[3], 'rb') as imgpath:
    x_test = np.frombuffer(
        imgpath.read(), np.uint8, offset=16).reshape(len(y_test), 28, 28)

  return (x_train, y_train), (x_test, y_test)

When calling this function:

(x_train,y_train),(x_test,y_test)=load_data_fromlocalpath('Your path')

Hi team, thanks for making this wonderful dataset.

Just want to share that I made a web based 3D interactive explorer of all the 70k images. It used UMAP to embed the images into a 3D space where users can freely navigate, or jump directly to a specific image by id.

Here is the link to the demo. Wondering if this would be an nice addition to the current explorations examples, and I am happy to make an PR if appropriate.

Best

Dear Fashion-MNIST Creator,

Thank you for your great contribution to our research society.

With E2E-3M, we have achieved a competitive result of 95.92% accuracy. More details can be found in the following links

https://arxiv.org/abs/2007.15161 https://github.com/leonlha/e2e-3m

Cheers! Phong