Introduction to the Scheduler Framework
Get introduced to the Kubernetes scheduler framework.
We'll cover the following...
The scheduler framework
For a well-designed open-source project, one distinguishing feature is its extensibility. This would help users and developers fulfill their own business needs easily without breaking upstream source codes and contribute back.
Kubernetes provides a series of ways to customize the scheduler, such as multiple schedulers and scheduler extenders. Multiple schedulers can have good performance, but they bring in high maintenance costs, resource competition problems, etc. It also has a steep developing curve—developers need to have a comprehensive understanding of the scheduling process. While the scheduler extender is easy to extend and maintain, it only provides limited extension points and has poor performance. To deal with this dilemma and provide a friendly developing experience, the Kubernetes community adopted a pluggable architecture (the scheduler framework) for the scheduler starting from v1.16.
The scheduler framework defines a variety of plugin APIs for extension points in the scheduling process. This design allows scheduling features to be implemented as independent plugins, while keeping the scheduling process lightweight, extensive, and maintainable. Below are all the registered in-tree scheduling plugins (lines 14–34). We can configure the default scheduler to enable or disable the use of certain plugins at runtime. We can also set custom plugin arguments for each plugin. This is configurable by a scheduling profile, which is part of a scheduler configuration.
// NewInTreeRegistry builds the registry with all the in-tree plugins.// A scheduler that runs out of tree plugins can register additional plugins// through the WithFrameworkOutOfTreeRegistry option.func NewInTreeRegistry() runtime.Registry {fts := plfeature.Features{EnableReadWriteOncePod: feature.DefaultFeatureGate.Enabled(features.ReadWriteOncePod),EnableVolumeCapacityPriority: feature.DefaultFeatureGate.Enabled(features.VolumeCapacityPriority),EnableMinDomainsInPodTopologySpread: feature.DefaultFeatureGate.Enabled(features.MinDomainsInPodTopologySpread),EnableNodeInclusionPolicyInPodTopologySpread: feature.DefaultFeatureGate.Enabled(features.NodeInclusionPolicyInPodTopologySpread),EnableMatchLabelKeysInPodTopologySpread: feature.DefaultFeatureGate.Enabled(features.MatchLabelKeysInPodTopologySpread),}return runtime.Registry{selectorspread.Name: selectorspread.New,imagelocality.Name: imagelocality.New,tainttoleration.Name: tainttoleration.New,nodename.Name: nodename.New,nodeports.Name: nodeports.New,nodeaffinity.Name: nodeaffinity.New,podtopologyspread.Name: runtime.FactoryAdapter(fts, podtopologyspread.New),nodeunschedulable.Name: nodeunschedulable.New,noderesources.Name: runtime.FactoryAdapter(fts, noderesources.NewFit),noderesources.BalancedAllocationName: runtime.FactoryAdapter(fts, noderesources.NewBalancedAllocation),volumebinding.Name: runtime.FactoryAdapter(fts, volumebinding.New),volumerestrictions.Name: runtime.FactoryAdapter(fts, volumerestrictions.New),volumezone.Name: volumezone.New,nodevolumelimits.CSIName: runtime.FactoryAdapter(fts, nodevolumelimits.NewCSI),nodevolumelimits.EBSName: runtime.FactoryAdapter(fts, nodevolumelimits.NewEBS),nodevolumelimits.GCEPDName: runtime.FactoryAdapter(fts, nodevolumelimits.NewGCEPD),nodevolumelimits.AzureDiskName: runtime.FactoryAdapter(fts, nodevolumelimits.NewAzureDisk),interpodaffinity.Name: interpodaffinity.New,queuesort.Name: queuesort.New,defaultbinder.Name: defaultbinder.New,defaultpreemption.Name: runtime.FactoryAdapter(fts, defaultpreemption.New),}}
Developers can implement their own plugins by implementing the APIs defined at each extension point. For each plugin, it can implement one or multiple interfaces of extension points, such as Filter and Score. This means that a plugin could be invoked once or multiple times during a scheduling process. The plugins can be informational or they help make a scheduling ...