I'm really really sorry. Because of my poor English skills, I didn't think "excuse me" meant contempt.I just wanted to ask one question.I am really sorry that I have brought you a bad mood。And I did have a problem when I run your source code that like "the surviving open question". Can you give a general analysis of the reasons? Thank you a thousand times, you're an angel.

Hey, first of, thanks for sharing, it is awesome!

I would just like to say that I tried to replicate your results and even though I had a lot of trouble with the attr package (because you were using almost the dev version, right?) I got around it.

This however seems like a problem I cant fix on my own.

Upon this command: ./train.py checkpoint-folder --all --hps dice_loss=10,n_epochs=70 as given in the README, I get this error:

Traceback (most recent call last):
  File "./train.py", line 704, in <module>
    main()
  File "./train.py", line 640, in main
    all_im_ids = list(utils.get_wkt_data())
  File "/root/sharedfolder/kaggle-dstl/utils.py", line 41, in get_wkt_data
    with open('./train_wkt_v4.csv') as f:
FileNotFoundError: [Errno 2] No such file or directory: './train_wkt_v4.csv'

However it first gives me some long parameter filled dict, if that helps. Any chance you might share this file? Even if it is private?

Position: train.py->main() arg('--model-path',type=Path) Question: Could you tell me what to fill in here? ( train_geojson_v3 ?) if is 'train_geojson_v3', it‘s error :No such file or directory: 'align_cache\6010_1_2_p.alignment'

first, i want very thanks the autor ,it`s a outstading project,howener ,there are some problem in it. it lead to the source can not work well.

i find two very serious problems.

A:the list is to explain the problem.

in train.py
` inputs.append(patch[:, m: -m, m: -m].astype(np.float32))
outputs.append(mask[:, m: -m, m: -m].astype(np.float32))

       if self.hps.needs_dist:
           dist_outputs.append(dist_mask[:, m: -m, m: -m].astype(np.float32))`
   return (torch.from_numpy(np.array(inputs)),
           torch.from_numpy(np.array(outputs)),
           torch.from_numpy(np.array(dist_outputs)))

the return 3th args, (dist_outputs) is null . so it is not transform to torch.
the error message is: RuntimeError: the given numpy array has zero-sized dimensions. Zero-sized dimensions are not supported in PyTorch

in models.py
@property def needs_dist(self): return (self.dist_loss != 0 or self.dist_dice_loss != 0 or self.dist_jaccard_loss != 0)
you can find "title 1" `s if-else decide by this function

however , the source code default this 3 value is 0
in models.py
dist_loss = attr.ib(default=0.0) dist_dice_loss = attr.ib(default=0.0) dist_jaccard_loss = attr.ib(default=0.0)

the web course is : $ ./train.py checkpoint-floder --all --hps dice_loss=10,n_epochs=70
so come up the "title 1" problem.

5. I don`t kown that how to moditfy the parameter in train.py (the main function)

B . the second problem in train.py _train_on_feeds()

` def _train_on_feeds(self, gen_batch, n_batches: int, no_mp: bool): losses = [[] for _ in range(self.hps.n_classes)] #self.hp.n_classes=10 jaccard_stats = self._jaccard_stats()

def log():
    logger.info(
        'Train loss: {loss:.3f}, Jaccard: {jaccard}, '
        'speed: {speed:,} patches/s'.format(
            loss=np.array(losses)[:, -log_step:].mean(),
            speed=int(len(losses[0]) * self.hps.batch_size / (t1 - t00)),
            jaccard=self._format_jaccard(jaccard_stats),
        ))

t0 = t00 = time.time()
log_step = 50
im_log_step = n_batches // log_step * log_step
map_ = (map if no_mp else
        partial(utils.imap_fixed_output_buffer, threads=4))
for i, (x, y, dist_y) in enumerate(map_(gen_batch, range(n_batches))):
    if losses[0] and i % log_step == 0:
        for cls, ls in zip(self.hps.classes, losses):
            self._log_value(
                'loss/cls-{}'.format(cls), np.mean(ls[-log_step:]))
        if self.hps.has_all_classes:
            self._log_value(
                'loss/cls-mean', np.mean([
                    l for ls in losses for l in ls[-log_step:]]))
        pred_y = self.net(self._var(x)).data.cpu()                         
        self._update_jaccard(jaccard_stats, y.numpy(), pred_y.numpy())
        self._log_jaccard(jaccard_stats)
        if i == im_log_step:
            self._log_im(
                x.numpy(), y.numpy(), dist_y.numpy(), pred_y.numpy())
    step_losses = self.train_step(x, y, dist_y)        
                                                        
    for ls, l in zip(losses, step_losses):
        ls.append(l)
    t1 = time.time()
    dt = t1 - t0
    if dt > 10:
        log()
        jaccard_stats = self._jaccard_stats()
        t0 = t1
if losses:
    log()`

the code : if losses[0] and i % log_step == 0: this if will never run.

in train.py arg('logdir', help='Path to log directory') arg('--hps', help='Change hyperparameters in k1=v1,k2=v2 format') arg('--all', action='store_true', help='Train on all images without validation') arg('--validation', choices=['random', 'stratified', 'square', 'custom'], default='custom', help='validation strategy') arg('--valid-only', action='store_true') arg('--only', help='Train on this image ids only (comma-separated) without validation') arg('--clean', action='store_true', help='Clean logdir') arg('--no-mp', action='store_true', help='Disable multiprocessing') arg('--model-path', type=Path)

excuse me, the last 'model-path' stands for what?

if you run the original source code ,you will find a very serious problem.

the list is to explain the problem.

1. in train.py ` inputs.append(patch[:, m: -m, m: -m].astype(np.float32)) outputs.append(mask[:, m: -m, m: -m].astype(np.float32))

          if self.hps.needs_dist:
              dist_outputs.append(dist_mask[:, m: -m, m: -m].astype(np.float32))`
      return (torch.from_numpy(np.array(inputs)),
              torch.from_numpy(np.array(outputs)),
              torch.from_numpy(np.array(dist_outputs)))
   

   the return 3th args, (dist_outputs) is null . so it is not transform to torch. the error message is: RuntimeError: the given numpy array has zero-sized dimensions. Zero-sized dimensions are not supported in PyTorch

2. in models.py @property def needs_dist(self): return (self.dist_loss != 0 or self.dist_dice_loss != 0 or self.dist_jaccard_loss != 0) you can find "title 1" `s if-else decide by this function

3. however , the source code default this 3 value is 0 in models.py dist_loss = attr.ib(default=0.0) dist_dice_loss = attr.ib(default=0.0) dist_jaccard_loss = attr.ib(default=0.0)

4. the web course is : $ ./train.py checkpoint-floder --all --hps dice_loss=10,n_epochs=70 so come up the "title 1" problem.

5. I don`t kown that how to solve this problem.

B . the second problem in train.py _train_on_feeds()

  `    def _train_on_feeds(self, gen_batch, n_batches: int, no_mp: bool):
    losses = [[] for _ in range(self.hps.n_classes)]      #self.hp.n_classes=10
    jaccard_stats = self._jaccard_stats()

    def log():
        logger.info(
            'Train loss: {loss:.3f}, Jaccard: {jaccard}, '
            'speed: {speed:,} patches/s'.format(
                loss=np.array(losses)[:, -log_step:].mean(),
                speed=int(len(losses[0]) * self.hps.batch_size / (t1 - t00)),
                jaccard=self._format_jaccard(jaccard_stats),
            ))

    t0 = t00 = time.time()
    log_step = 50
    im_log_step = n_batches // log_step * log_step
    map_ = (map if no_mp else
            partial(utils.imap_fixed_output_buffer, threads=4))
    for i, (x, y, dist_y) in enumerate(map_(gen_batch, range(n_batches))):
        if losses[0] and i % log_step == 0:
            for cls, ls in zip(self.hps.classes, losses):
                self._log_value(
                    'loss/cls-{}'.format(cls), np.mean(ls[-log_step:]))
            if self.hps.has_all_classes:
                self._log_value(
                    'loss/cls-mean', np.mean([
                        l for ls in losses for l in ls[-log_step:]]))
            pred_y = self.net(self._var(x)).data.cpu()                         
            self._update_jaccard(jaccard_stats, y.numpy(), pred_y.numpy())
            self._log_jaccard(jaccard_stats)
            if i == im_log_step:
                self._log_im(
                    x.numpy(), y.numpy(), dist_y.numpy(), pred_y.numpy())
        step_losses = self.train_step(x, y, dist_y)        
                                                            
        for ls, l in zip(losses, step_losses):
            ls.append(l)
        t1 = time.time()
        dt = t1 - t0
        if dt > 10:
            log()
            jaccard_stats = self._jaccard_stats()
            t0 = t1
    if losses:
        log()`

the code : if losses[0] and i % log_step == 0: this if will never run.

hi,

should it be $ ./train.py checkpoint-folder --all --hps dist_loss=10,n_epochs=70 instead of $ ./train.py checkpoint-folder --all --hps dice_loss=10,n_epochs=70

Thank you!

made a mistake when I ran the train.py.

RuntimeError: the given numpy array has zero-sized dimensions. Zero-sized dimensions are not supported in PyTorch