Neural network innovations