An extension of the inner loop vectorizer that creates a skeleton for a vectorized loop that has its epilogue (residual) also vectorized. More...

Inheritance diagram for llvm::InnerLoopAndEpilogueVectorizer:

[legend]

Public Member Functions
	InnerLoopAndEpilogueVectorizer (Loop OrigLoop, PredicatedScalarEvolution &PSE, LoopInfo LI, DominatorTree DT, const TargetTransformInfo TTI, AssumptionCache AC, EpilogueLoopVectorizationInfo &EPI, LoopVectorizationCostModel CM, BlockFrequencyInfo BFI, ProfileSummaryInfo PSI, GeneratedRTChecks &Checks, VPlan &Plan, ElementCount VecWidth, ElementCount MinProfitableTripCount, unsigned UnrollFactor)
Public Member Functions inherited from llvm::InnerLoopVectorizer
	InnerLoopVectorizer (Loop OrigLoop, PredicatedScalarEvolution &PSE, LoopInfo LI, DominatorTree DT, const TargetTransformInfo TTI, AssumptionCache AC, ElementCount VecWidth, unsigned UnrollFactor, LoopVectorizationCostModel CM, BlockFrequencyInfo BFI, ProfileSummaryInfo PSI, GeneratedRTChecks &RTChecks, VPlan &Plan)
virtual	~InnerLoopVectorizer ()=default
virtual BasicBlock *	createVectorizedLoopSkeleton ()
	Creates a basic block for the scalar preheader.
void	fixVectorizedLoop (VPTransformState &State)
	Fix the vectorized code, taking care of header phi's, and more.
void	fixNonInductionPHIs (VPTransformState &State)
	Fix the non-induction PHIs in `Plan`.
Value *	getTripCount () const
	Returns the original loop trip count.
void	setTripCount (Value *TC)
	Used to set the trip count after ILV's construction and after the preheader block has been executed.

Public Attributes
EpilogueLoopVectorizationInfo &	EPI
	Holds and updates state information required to vectorize the main loop and its epilogue in two separate passes.

Protected Attributes
ElementCount	MinProfitableTripCount
Protected Attributes inherited from llvm::InnerLoopVectorizer
Loop *	OrigLoop
	The original loop.
PredicatedScalarEvolution &	PSE
	A wrapper around ScalarEvolution used to add runtime SCEV checks.
LoopInfo *	LI
	Loop Info.
DominatorTree *	DT
	Dominator Tree.
const TargetTransformInfo *	TTI
	Target Transform Info.
AssumptionCache *	AC
	Assumption Cache.
ElementCount	VF
	The vectorization SIMD factor to use.
unsigned	UF
	The vectorization unroll factor to use.
IRBuilder	Builder
	The builder that we use.
Value *	TripCount = nullptr
	Trip count of the original loop.
LoopVectorizationCostModel *	Cost
	The profitablity analysis.
BlockFrequencyInfo *	BFI
	BFI and PSI are used to check for profile guided size optimizations.
ProfileSummaryInfo *	PSI
GeneratedRTChecks &	RTChecks
	Structure to hold information about generated runtime checks, responsible for cleaning the checks, if vectorization turns out unprofitable.
VPlan &	Plan
VPBasicBlock *	VectorPHVPBB
	The vector preheader block of `Plan`, used as target for check blocks introduced during skeleton creation.

Additional Inherited Members
Protected Member Functions inherited from llvm::InnerLoopVectorizer
BasicBlock *	createScalarPreheader (StringRef Prefix)
	Create and return a new IR basic block for the scalar preheader whose name is prefixed with `Prefix`.
virtual void	printDebugTracesAtStart ()
	Allow subclasses to override and print debug traces before/after vplan execution, when trace information is requested.
virtual void	printDebugTracesAtEnd ()

Detailed Description

An extension of the inner loop vectorizer that creates a skeleton for a vectorized loop that has its epilogue (residual) also vectorized.

The idea is to run the vplan on a given loop twice, firstly to setup the skeleton and vectorize the main loop, and secondly to complete the skeleton from the first step and vectorize the epilogue. This is achieved by deriving two concrete strategy classes from this base class and invoking them in succession from the loop vectorizer planner.

Definition at line 627 of file LoopVectorize.cpp.

Constructor & Destructor Documentation

◆ InnerLoopAndEpilogueVectorizer()

llvm::InnerLoopAndEpilogueVectorizer::InnerLoopAndEpilogueVectorizer	(	Loop *	OrigLoop,
		PredicatedScalarEvolution &	PSE,
		LoopInfo *	LI,
		DominatorTree *	DT,
		const TargetTransformInfo *	TTI,
		AssumptionCache *	AC,
		EpilogueLoopVectorizationInfo &	EPI,
		LoopVectorizationCostModel *	CM,
		BlockFrequencyInfo *	BFI,
		ProfileSummaryInfo *	PSI,
		GeneratedRTChecks &	Checks,
		VPlan &	Plan,
		ElementCount	VecWidth,
		ElementCount	MinProfitableTripCount,
		unsigned	UnrollFactor )

inline

Definition at line 629 of file LoopVectorize.cpp.

References llvm::InnerLoopVectorizer::AC, llvm::InnerLoopVectorizer::BFI, llvm::InnerLoopVectorizer::DT, EPI, llvm::InnerLoopVectorizer::InnerLoopVectorizer(), llvm::InnerLoopVectorizer::LI, MinProfitableTripCount, llvm::InnerLoopVectorizer::OrigLoop, llvm::InnerLoopVectorizer::Plan, llvm::InnerLoopVectorizer::PSE, llvm::InnerLoopVectorizer::PSI, and llvm::InnerLoopVectorizer::TTI.

Referenced by llvm::EpilogueVectorizerEpilogueLoop::EpilogueVectorizerEpilogueLoop(), and llvm::EpilogueVectorizerMainLoop::EpilogueVectorizerMainLoop().

Member Data Documentation

◆ EPI

EpilogueLoopVectorizationInfo& llvm::InnerLoopAndEpilogueVectorizer::EPI

Holds and updates state information required to vectorize the main loop and its epilogue in two separate passes.

This setup helps us avoid regenerating and recomputing runtime safety checks. It also helps us to shorten the iteration-count-check path length for the cases where the iteration count of the loop is so small that the main vector loop is completely skipped.

Definition at line 646 of file LoopVectorize.cpp.

◆ MinProfitableTripCount

ElementCount llvm::InnerLoopAndEpilogueVectorizer::MinProfitableTripCount

protected

Definition at line 649 of file LoopVectorize.cpp.

Referenced by llvm::EpilogueVectorizerMainLoop::createIterationCountCheck(), llvm::EpilogueVectorizerMainLoop::emitIterationCountCheck(), and InnerLoopAndEpilogueVectorizer().

The documentation for this class was generated from the following file:

lib/Transforms/Vectorize/LoopVectorize.cpp

Public Member Functions

Public Attributes