123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718 |
- PSCI OS-initiated mode
- ======================
- :Author: Maulik Shah & Wing Li
- :Organization: Qualcomm Innovation Center, Inc. & Google LLC
- :Contact: Maulik Shah <quic_mkshah@quicinc.com> & Wing Li <wingers@google.com>
- :Status: Accepted
- .. contents:: Table of Contents
- Introduction
- ------------
- Power state coordination
- ^^^^^^^^^^^^^^^^^^^^^^^^
- A power domain topology is a logical hierarchy of power domains in a system that
- arises from the physical dependencies between power domains.
- Local power states describe power states for an individual node, and composite
- power states describe the combined power states for an individual node and its
- parent node(s).
- Entry into low-power states for a topology node above the core level requires
- coordinating its children nodes. For example, in a system with a power domain
- that encompasses a shared cache, and a separate power domain for each core that
- uses the shared cache, the core power domains must be powered down before the
- shared cache power domain can be powered down.
- PSCI supports two modes of power state coordination: platform-coordinated and
- OS-initiated.
- Platform-coordinated
- ~~~~~~~~~~~~~~~~~~~~
- Platform-coordinated mode is the default mode of power state coordination, and
- is currently the only supported mode in TF-A.
- In platform-coordinated mode, the platform is responsible for coordinating power
- states, and chooses the deepest power state for a topology node that can be
- tolerated by its children.
- OS-initiated
- ~~~~~~~~~~~~
- OS-initiated mode is optional.
- In OS-initiated mode, the calling OS is responsible for coordinating power
- states, and may request for a topology node to enter a low-power state when
- its last child enters the low-power state.
- Motivation
- ----------
- There are two reasons why OS-initiated mode might be a more suitable option than
- platform-coordinated mode for a platform.
- Scalability
- ^^^^^^^^^^^
- In platform-coordinated mode, each core independently selects their own local
- power states, and doesn't account for composite power states that are shared
- between cores.
- In OS-initiated mode, the OS has knowledge of the next wakeup event for each
- core, and can have more precise control over the entry, exit, and wakeup
- latencies when deciding if a composite power state (e.g. for a cluster) is
- appropriate. This is especially important for multi-cluster SMP systems and
- heterogeneous systems like big.LITTLE, where different processor types can have
- different power efficiencies.
- Simplicity
- ^^^^^^^^^^
- In platform-coordinated mode, the OS doesn't have visibility when the last core
- at a power level enters a low-power state. If the OS wants to perform last man
- activity (e.g. powering off a shared resource when it is no longer needed), it
- would have to communicate with an API side channel to know when it can do so.
- This could result in a design smell where the platform is using
- platform-coordinated mode when it should be using OS-initiated mode instead.
- In OS-initiated mode, the OS can perform last man activity if it selects a
- composite power state when the last core enters a low-power state. This
- eliminates the need for a side channel, and uses the well documented API between
- the OS and the platform.
- Current vendor implementations and workarounds
- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- * STMicroelectronics
- * For their ARM32 platforms, they're using OS-initiated mode implemented in
- OP-TEE.
- * For their future ARM64 platforms, they are interested in using OS-initiated
- mode in TF-A.
- * Qualcomm
- * For their mobile platforms, they're using OS-initiated mode implemented in
- their own custom secure monitor firmware.
- * For their Chrome OS platforms, they're using platform-coordinated mode in
- TF-A with custom driver logic to perform last man activity.
- * Google
- * They're using platform-coordinated mode in TF-A with custom driver logic to
- perform last man activity.
- Both Qualcomm and Google would like to be able to use OS-initiated mode in TF-A
- in order to simplify custom driver logic.
- Requirements
- ------------
- PSCI_FEATURES
- ^^^^^^^^^^^^^
- PSCI_FEATURES is for checking whether or not a PSCI function is implemented and
- what its properties are.
- .. c:macro:: PSCI_FEATURES
- :param func_id: 0x8400_000A.
- :param psci_func_id: the function ID of a PSCI function.
- :retval NOT_SUPPORTED: if the function is not implemented.
- :retval feature flags associated with the function: if the function is
- implemented.
- CPU_SUSPEND feature flags
- ~~~~~~~~~~~~~~~~~~~~~~~~~
- * Reserved, bits[31:2]
- * Power state parameter format, bit[1]
- * A value of 0 indicates the original format is used.
- * A value of 1 indicates the extended format is used.
- * OS-initiated mode, bit[0]
- * A value of 0 indicates OS-initiated mode is not supported.
- * A value of 1 indicates OS-initiated mode is supported.
- See sections 5.1.14 and 5.15 of the PSCI spec (DEN0022D.b) for more details.
- PSCI_SET_SUSPEND_MODE
- ^^^^^^^^^^^^^^^^^^^^^
- PSCI_SET_SUSPEND_MODE is for switching between the two different modes of power
- state coordination.
- .. c:macro:: PSCI_SET_SUSPEND_MODE
- :param func_id: 0x8400_000F.
- :param mode: 0 indicates platform-coordinated mode, 1 indicates OS-initiated
- mode.
- :retval SUCCESS: if the request is successful.
- :retval NOT_SUPPORTED: if OS-initiated mode is not supported.
- :retval INVALID_PARAMETERS: if the requested mode is not a valid value (0 or
- 1).
- :retval DENIED: if the cores are not in the correct state.
- Switching from platform-coordinated to OS-initiated is only allowed if the
- following conditions are met:
- * All cores are in one of the following states:
- * Running.
- * Off, through a call to CPU_OFF or not yet booted.
- * Suspended, through a call to CPU_DEFAULT_SUSPEND.
- * None of the cores has called CPU_SUSPEND since the last change of mode or
- boot.
- Switching from OS-initiated to platform-coordinated is only allowed if all cores
- other than the calling core are off, either through a call to CPU_OFF or not yet
- booted.
- If these conditions are not met, the PSCI implementation must return DENIED.
- See sections 5.1.19 and 5.20 of the PSCI spec (DEN0022D.b) for more details.
- CPU_SUSPEND
- ^^^^^^^^^^^
- CPU_SUSPEND is for moving a topology node into a low-power state.
- .. c:macro:: CPU_SUSPEND
- :param func_id: 0xC400_0001.
- :param power_state: the requested low-power state to enter.
- :param entry_point_address: the address at which the core must resume
- execution following wakeup from a powerdown state.
- :param context_id: this field specifies a pointer to the saved context that
- must be restored on a core following wakeup from a powerdown state.
- :retval SUCCESS: if the request is successful.
- :retval INVALID_PARAMETERS: in OS-initiated mode, this error is returned when
- a low-power state is requested for a topology node above the core level,
- and at least one of the node's children is in a local low-power state
- that is incompatible with the request.
- :retval INVALID_ADDRESS: if the entry_point_address argument is invalid.
- :retval DENIED: only in OS-initiated mode; this error is returned when a
- low-power state is requested for a topology node above the core level,
- and at least one of the node's children is running, i.e. not in a
- low-power state.
- In platform-coordinated mode, the PSCI implementation coordinates requests from
- all cores to determine the deepest power state to enter.
- In OS-initiated mode, the calling OS is making an explicit request for a
- specific power state, as opposed to expressing a vote. The PSCI implementation
- must comply with the request, unless the request is not consistent with the
- implementation's view of the system's state, in which case, the implementation
- must return INVALID_PARAMETERS or DENIED.
- See sections 5.1.2 and 5.4 of the PSCI spec (DEN0022D.b) for more details.
- Power state formats
- ~~~~~~~~~~~~~~~~~~~
- Original format
- * Power Level, bits[25:24]
- * The requested level in the power domain topology to enter a low-power
- state.
- * State Type, bit[16]
- * A value of 0 indicates a standby or retention state.
- * A value of 1 indicates a powerdown state.
- * State ID, bits[15:0]
- * Field to specify the requested composite power state.
- * The state ID encodings must uniquely describe every possible composite
- power state.
- * In OS-initiated mode, the state ID encoding must allow expressing the
- power level at which the calling core is the last to enter a powerdown
- state.
- Extended format
- * State Type, bit[30]
- * State ID, bits[27:0]
- Races in OS-initiated mode
- ~~~~~~~~~~~~~~~~~~~~~~~~~~
- In OS-initiated mode, there are race windows where the OS's view and
- implementation's view of the system's state differ. It is possible for the OS to
- make requests that are invalid given the implementation's view of the system's
- state. For example, the OS might request a powerdown state for a node from one
- core, while at the same time, the implementation observes that another core in
- that node is powering up.
- To address potential race conditions in power state requests:
- * The calling OS must specify in each CPU_SUSPEND request the deepest power
- level for which it sees the calling core as the last running core (last man).
- This is required even if the OS doesn't want the node at that power level to
- enter a low-power state.
- * The implementation must validate that the requested power states in the
- CPU_SUSPEND request are consistent with the system's state, and that the
- calling core is the last core running at the requested power level, or deny
- the request otherwise.
- See sections 4.2.3.2, 6.2, and 6.3 of the PSCI spec (DEN0022D.b) for more
- details.
- Caveats
- -------
- CPU_OFF
- ^^^^^^^
- CPU_OFF is always platform-coordinated, regardless of whether the power state
- coordination mode for suspend is platform-coordinated or OS-initiated. If all
- cores in a topology node call CPU_OFF, the last core will power down the node.
- In OS-initiated mode, if a subset of the cores in a topology node has called
- CPU_OFF, the last running core may call CPU_SUSPEND to request a powerdown state
- at or above that node's power level.
- See section 5.5.2 of the PSCI spec (DEN0022D.b) for more details.
- Implementation
- --------------
- Current implementation of platform-coordinated mode
- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- Platform-coordinated is currently the only supported power state coordination
- mode in TF-A.
- The functions of interest in the ``psci_cpu_suspend`` call stack are as follows:
- * ``psci_validate_power_state``
- * This function calls a platform specific ``validate_power_state`` handler,
- which takes the ``power_state`` parameter, and updates the ``state_info``
- object with the requested states for each power level.
- * ``psci_find_target_suspend_lvl``
- * This function takes the ``state_info`` object containing the requested power
- states for each power level, and returns the deepest power level that was
- requested to enter a low power state, i.e. the target power level.
- * ``psci_do_state_coordination``
- * This function takes the target power level and the ``state_info`` object
- containing the requested power states for each power level, and updates the
- ``state_info`` object with the coordinated target power state for each
- level.
- * ``pwr_domain_suspend``
- * This is a platform specific handler that takes the ``state_info`` object
- containing the target power states for each power level, and transitions
- each power level to the specified power state.
- Proposed implementation of OS-initiated mode
- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- To add support for OS-initiated mode, the following changes are proposed:
- * Add a boolean build option ``PSCI_OS_INIT_MODE`` for a platform to enable
- optional support for PSCI OS-initiated mode. This build option defaults to 0.
- .. note::
- If ``PSCI_OS_INIT_MODE=0``, the following changes will not be compiled into
- the build.
- * Update ``psci_features`` to return 1 in bit[0] to indicate support for
- OS-initiated mode for CPU_SUSPEND.
- * Define a ``suspend_mode`` enum: ``PLAT_COORD`` and ``OS_INIT``.
- * Define a ``psci_suspend_mode`` global variable with a default value of
- ``PLAT_COORD``.
- * Implement a new function handler ``psci_set_suspend_mode`` for
- PSCI_SET_SUSPEND_MODE.
- * Since ``psci_validate_power_state`` calls a platform specific
- ``validate_power_state`` handler, the platform implementation should populate
- the ``state_info`` object based on the state ID from the given ``power_state``
- parameter.
- * ``psci_find_target_suspend_lvl`` remains unchanged.
- * Implement a new function ``psci_validate_state_coordination`` that ensures the
- request satisfies the following conditions, and denies any requests
- that don't:
- * The requested power states for each power level are consistent with the
- system's state
- * The calling core is the last core running at the requested power level
- This function differs from ``psci_do_state_coordination`` in that:
- * The ``psci_req_local_pwr_states`` map is not modified if the request were to
- be denied
- * The ``state_info`` argument is never modified since it contains the power
- states requested by the calling OS
- * Update ``psci_cpu_suspend_start`` to do the following:
- * If ``PSCI_SUSPEND_MODE`` is ``PLAT_COORD``, call
- ``psci_do_state_coordination``.
- * If ``PSCI_SUSPEND_MODE`` is ``OS_INIT``, call
- ``psci_validate_state_coordination``. If validation fails, propagate the
- error up the call stack.
- * Add a new optional member ``pwr_domain_validate_suspend`` to
- ``plat_psci_ops_t`` to allow the platform to optionally perform validations
- based on hardware states.
- * The platform specific ``pwr_domain_suspend`` handler remains unchanged.
- .. image:: ../resources/diagrams/psci-osi-mode.png
- Testing
- -------
- The proposed patches can be found at
- https://review.trustedfirmware.org/q/topic:psci-osi.
- Testing on FVP and Google platforms
- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- The proposed patches add a new CPU Suspend in OSI mode test suite to TF-A Tests.
- This has been enabled and verified on the FVP_Base_RevC-2xAEMvA platform and
- Google platforms, and excluded from all other platforms via the build option
- ``PLAT_TESTS_SKIP_LIST``.
- Testing on STM32MP15
- ^^^^^^^^^^^^^^^^^^^^
- The proposed patches have been tested and verified on the STM32MP15 platform,
- which has a single cluster with 2 CPUs, by Gabriel Fernandez
- <gabriel.fernandez@st.com> from STMicroelectronics with this device tree
- configuration:
- .. code-block:: devicetree
- cpus {
-
-
- cpu0: cpu@0 {
- device_type = "cpu";
- compatible = "arm,cortex-a7";
- reg = <0>;
- enable-method = "psci";
- power-domains = <&CPU_PD0>;
- power-domain-names = "psci";
- };
- cpu1: cpu@1 {
- device_type = "cpu";
- compatible = "arm,cortex-a7";
- reg = <1>;
- enable-method = "psci";
- power-domains = <&CPU_PD1>;
- power-domain-names = "psci";
- };
- idle-states {
- cpu_retention: cpu-retention {
- compatible = "arm,idle-state";
- arm,psci-suspend-param = <0x00000001>;
- entry-latency-us = <130>;
- exit-latency-us = <620>;
- min-residency-us = <700>;
- local-timer-stop;
- };
- };
- domain-idle-states {
- CLUSTER_STOP: core-power-domain {
- compatible = "domain-idle-state";
- arm,psci-suspend-param = <0x01000001>;
- entry-latency-us = <230>;
- exit-latency-us = <720>;
- min-residency-us = <2000>;
- local-timer-stop;
- };
- };
- };
- psci {
- compatible = "arm,psci-1.0";
- method = "smc";
- CPU_PD0: power-domain-cpu0 {
-
- power-domains = <&pd_core>;
- domain-idle-states = <&cpu_retention>;
- };
- CPU_PD1: power-domain-cpu1 {
-
- power-domains = <&pd_core>;
- domain-idle-states = <&cpu_retention>;
- };
- pd_core: power-domain-cluster {
-
- domain-idle-states = <&CLUSTER_STOP>;
- };
- };
- Testing on Qualcomm SC7280
- ^^^^^^^^^^^^^^^^^^^^^^^^^^
- The proposed patches have been tested and verified on the SC7280 platform by
- Maulik Shah <quic_mkshah@quicinc.com> from Qualcomm with this device tree
- configuration:
- .. code-block:: devicetree
- cpus {
-
-
- CPU0: cpu@0 {
- device_type = "cpu";
- compatible = "arm,kryo";
- reg = <0x0 0x0>;
- enable-method = "psci";
- power-domains = <&CPU_PD0>;
- power-domain-names = "psci";
- };
- CPU1: cpu@100 {
- device_type = "cpu";
- compatible = "arm,kryo";
- reg = <0x0 0x100>;
- enable-method = "psci";
- power-domains = <&CPU_PD1>;
- power-domain-names = "psci";
- };
- CPU2: cpu@200 {
- device_type = "cpu";
- compatible = "arm,kryo";
- reg = <0x0 0x200>;
- enable-method = "psci";
- power-domains = <&CPU_PD2>;
- power-domain-names = "psci";
- };
- CPU3: cpu@300 {
- device_type = "cpu";
- compatible = "arm,kryo";
- reg = <0x0 0x300>;
- enable-method = "psci";
- power-domains = <&CPU_PD3>;
- power-domain-names = "psci";
- }
- CPU4: cpu@400 {
- device_type = "cpu";
- compatible = "arm,kryo";
- reg = <0x0 0x400>;
- enable-method = "psci";
- power-domains = <&CPU_PD4>;
- power-domain-names = "psci";
- };
- CPU5: cpu@500 {
- device_type = "cpu";
- compatible = "arm,kryo";
- reg = <0x0 0x500>;
- enable-method = "psci";
- power-domains = <&CPU_PD5>;
- power-domain-names = "psci";
- };
- CPU6: cpu@600 {
- device_type = "cpu";
- compatible = "arm,kryo";
- reg = <0x0 0x600>;
- enable-method = "psci";
- power-domains = <&CPU_PD6>;
- power-domain-names = "psci";
- };
- CPU7: cpu@700 {
- device_type = "cpu";
- compatible = "arm,kryo";
- reg = <0x0 0x700>;
- enable-method = "psci";
- power-domains = <&CPU_PD7>;
- power-domain-names = "psci";
- };
- idle-states {
- entry-method = "psci";
- LITTLE_CPU_SLEEP_0: cpu-sleep-0-0 {
- compatible = "arm,idle-state";
- idle-state-name = "little-power-down";
- arm,psci-suspend-param = <0x40000003>;
- entry-latency-us = <549>;
- exit-latency-us = <901>;
- min-residency-us = <1774>;
- local-timer-stop;
- };
- LITTLE_CPU_SLEEP_1: cpu-sleep-0-1 {
- compatible = "arm,idle-state";
- idle-state-name = "little-rail-power-down";
- arm,psci-suspend-param = <0x40000004>;
- entry-latency-us = <702>;
- exit-latency-us = <915>;
- min-residency-us = <4001>;
- local-timer-stop;
- };
- BIG_CPU_SLEEP_0: cpu-sleep-1-0 {
- compatible = "arm,idle-state";
- idle-state-name = "big-power-down";
- arm,psci-suspend-param = <0x40000003>;
- entry-latency-us = <523>;
- exit-latency-us = <1244>;
- min-residency-us = <2207>;
- local-timer-stop;
- };
- BIG_CPU_SLEEP_1: cpu-sleep-1-1 {
- compatible = "arm,idle-state";
- idle-state-name = "big-rail-power-down";
- arm,psci-suspend-param = <0x40000004>;
- entry-latency-us = <526>;
- exit-latency-us = <1854>;
- min-residency-us = <5555>;
- local-timer-stop;
- };
- };
- domain-idle-states {
- CLUSTER_SLEEP_0: cluster-sleep-0 {
- compatible = "arm,idle-state";
- idle-state-name = "cluster-power-down";
- arm,psci-suspend-param = <0x40003444>;
- entry-latency-us = <3263>;
- exit-latency-us = <6562>;
- min-residency-us = <9926>;
- local-timer-stop;
- };
- };
- };
- psci {
- compatible = "arm,psci-1.0";
- method = "smc";
- CPU_PD0: cpu0 {
-
- power-domains = <&CLUSTER_PD>;
- domain-idle-states = <&LITTLE_CPU_SLEEP_0 &LITTLE_CPU_SLEEP_1>;
- };
- CPU_PD1: cpu1 {
-
- power-domains = <&CLUSTER_PD>;
- domain-idle-states = <&LITTLE_CPU_SLEEP_0 &LITTLE_CPU_SLEEP_1>;
- };
- CPU_PD2: cpu2 {
-
- power-domains = <&CLUSTER_PD>;
- domain-idle-states = <&LITTLE_CPU_SLEEP_0 &LITTLE_CPU_SLEEP_1>;
- };
- CPU_PD3: cpu3 {
-
- power-domains = <&CLUSTER_PD>;
- domain-idle-states = <&LITTLE_CPU_SLEEP_0 &LITTLE_CPU_SLEEP_1>;
- };
- CPU_PD4: cpu4 {
-
- power-domains = <&CLUSTER_PD>;
- domain-idle-states = <&BIG_CPU_SLEEP_0 &BIG_CPU_SLEEP_1>;
- };
- CPU_PD5: cpu5 {
-
- power-domains = <&CLUSTER_PD>;
- domain-idle-states = <&BIG_CPU_SLEEP_0 &BIG_CPU_SLEEP_1>;
- };
- CPU_PD6: cpu6 {
-
- power-domains = <&CLUSTER_PD>;
- domain-idle-states = <&BIG_CPU_SLEEP_0 &BIG_CPU_SLEEP_1>;
- };
- CPU_PD7: cpu7 {
-
- power-domains = <&CLUSTER_PD>;
- domain-idle-states = <&BIG_CPU_SLEEP_0 &BIG_CPU_SLEEP_1>;
- };
- CLUSTER_PD: cpu-cluster0 {
-
- domain-idle-states = <&CLUSTER_SLEEP_0>;
- };
- };
- Comparisons on Qualcomm SC7280
- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- CPUIdle states
- ~~~~~~~~~~~~~~
- * 8 CPUs, 1 L3 cache
- * Platform-coordinated mode
- * CPUIdle states
- * State0 - WFI
- * State1 - Core collapse
- * State2 - Rail collapse
- * State3 - L3 cache off and system resources voted off
- * OS-initiated mode
- * CPUIdle states
- * State0 - WFI
- * State1 - Core collapse
- * State2 - Rail collapse
- * Cluster domain idle state
- * State3 - L3 cache off and system resources voted off
- .. image:: ../resources/diagrams/psci-flattened-vs-hierarchical-idle-states.png
- Results
- ~~~~~~~
- * The following stats have been captured with fixed CPU frequencies from the use
- case of 10 seconds of device idle with the display turned on and Wi-Fi and
- modem turned off.
- * Count refers to the number of times a CPU or cluster entered power collapse.
- * Residency refers to the time in seconds a CPU or cluster stayed in power
- collapse.
- * The results are an average of 3 iterations of actual counts and residencies.
- .. image:: ../resources/diagrams/psci-pc-mode-vs-osi-mode.png
- OS-initiated mode was able to scale better than platform-coordinated mode for
- multiple CPUs. The count and residency results for state3 (i.e. a cluster domain
- idle state) in OS-initiated mode for multiple CPUs were much closer to the
- results for a single CPU than in platform-coordinated mode.
- --------------
- *Copyright (c) 2023, Arm Limited and Contributors. All rights reserved.*
|