TensorFlow:  1.8
Model:       resnet50
Dataset:     imagenet
Mode:        training
SingleSess:  False
Batch size:  256 global
             32 per device
Num batches: 5004
Num epochs:  1.00
Devices:     ['/job:worker/replica:0/task:0/gpu:0', '/job:worker/replica:0/task:0/gpu:1', '/job:worker/replica:0/task:0/gpu:2', '/job:worker/replica:0/task:0/gpu:3']
Data format: NCHW
Layout optimizer: False
Optimizer:   sgd
Variables:   distributed_replicated
Sync:        True
==========
Generating model
