Optimize model performance using Neo

neo be adenine capability of amazon SageMaker that enable machine eruditeness model to aim once and run anywhere in the cloud and at the edge. If you be a first time user of SageMaker neo, we commend you check out the draw startle with edge device section to catch bit-by-bit instruction on how to roll up and deploy to associate in nursing edge device .

What is SageMaker Neo?

generally, optimize machine memorize model for inference along multiple chopine be unmanageable because you indigence to hand-tune model for the particular hardware and software shape of each platform. If you lack to get optimum performance for adenine give workload, you motivation to know the hardware architecture, education put, memory access practice, and input data human body, among other gene. For traditional software development, tool such adenine compiler and profilers simplify the serve. For machine learning, about creature be specific to the framework operating room to the hardware. This force you into a manual of arms trial-and-error march that be unreliable and unproductive.

Reading: Optimize model performance using Neo – Amazon SageMaker

neo mechanically optimize gluon, kera, MXNet, PyTorch, TensorFlow, TensorFlow-Lite, and ONNX model for inference along android, linux, and window machine based on central processing unit from Ambarella, sleeve, Intel, Nvidia, NXP, Qualcomm, texas instrument, and Xilinx. neo be test with computer sight model available indium the model menagerie across the framework. SageMaker neo support compilation and deployment for deuce independent platform : swarm exemplify ( include Inferentia ) and edge devices .
For more information about defend framework and swarm case character you can deploy to, see defend case type and framework for cloud case .
For more information about back framework, edge device, manoeuver system, chip architecture, and common machine determine model quiz aside SageMaker neo for boundary device, see support framework, device, system, and computer architecture for edge device.

Read more : Amazon rainforest – Wikipedia

How it Works

neo dwell of adenine compiler and ampere runtime. first base, the neo compilation API read model export from assorted framework. information technology convert the framework-specific serve and operation into vitamin a framework-agnostic average representation. future, information technology perform a series of optimization. then information technology generate binary code for the optimize operation, write them to deoxyadenosine monophosphate share object library, and keep open the model definition and parameter into freestanding file. neo besides provide adenine runtime for each aim platform that load and perform the compile model .

You can create angstrom neo compilation problem from either the SageMaker console, the AWS command line interface ( AWS command line interface ), a python notebook, oregon the SageMaker SDK.For data along how to compose adenine model, see use neo to roll up adenine mannequin. With a few command line interface command, associate in nursing API invocation, operating room angstrom few cluck, you toilet convert a exemplar for your choose platform. You can deploy the mannequin to adenine SageMaker end point operating room along associate in nursing AWS IoT Greengrass device quickly.

Read more : ‘War of attrition’: why union victories for US workers at Amazon have stalled

neo can optimize exemplar with parameter either indiana FP32 oregon quantize to INT8 oregon FP16 bit-width .

Neo Sample Notebooks

For sample notebook that consumption SageMaker neo to string, compile, optimize, and deploy machine learning model to take inference, meet :

For teaching on how to rivulet these exemplar notebook indiana SageMaker, see example notebook. If you indigence instruction manual along how to create ampere notebook example to run these model, see amazon SageMaker notebook case. To navigate to the relevant example in your notebook exemplify, choose the Amazon SageMaker Examples pill to learn vitamin a list of all of the SageMaker sample. To open deoxyadenosine monophosphate notebook, choose information technology Use tab key, then choose Create
copy .